how science has been blinded by the Rothschild report (1971) and the purpose of government-funded R&D
😈💩👎
Palgrave Communications volume 2, Article number: 16053 (2016) Cite this article
Abstract
In 1971, (Lord) Victor Rothschild, reported to Government on “The Organisation and Management of Government R&D” and how government could become (in his words) a customer for research contracted from the Research Councils and other sources. Rothschild’s thinking implied that management of R&D by “customer” Departments would bring an understanding of research outputs. He proposed the transfer of applied science funds from Research Councils to Government Departments, providing each with a Chief Scientist as proxy customer for research to be commissioned on a “customer/contractor” basis. The Government largely adopted his proposals in 1972 and implemented them in 1974. The Rothschild reforms and the upheavals they brought were controversial at the time, though now in some instances, reversed and otherwise either forgotten or buried in unconscious assumptions. However, the Rothschild framework still underpins important assumptions about Departmental relationships with the science community, which in my view adversely affected the access of Government to expert advice. From the viewpoint of a participant in Government R&D management through this period, I explore the immediate response and the post-1980 history of the Rothschild reforms, discuss the way in which research commissioning became such a heavy task as to impede analysis and advice delivery, and consider alternative approaches, such as the “science broker” model. This article is published as part of a collection on scientific advice to governments.
Lord Rothschild’s report to GovernmentFootnote1 on “The Organisation and Management of Government R&D” came out in 1971, a year before I graduated. Its recommendations were largely accepted in 1972 and brought into force in 1974; I became a scientific Civil Servant the following year. In one way and another, my career was dominated by implementation of Rothschild’s ideas though, by the time I retired from the civil service at the end of 2012 (as Deputy CSA at the Department for Environment, Food and Rural Affairs, hereafter Defra), the report itself was, at least in Government, largely forgotten. I reflect here on my practitioner’s experience of implementing Rothschild’s principles and their implications, with particular respect to their effect on “evidence based” policy-making.
Though it caused an enormous furore at the time of its publication (which I outline below), Rothschild’s report is remembered, if at all, for three main reasons:
as the origin of the application of the customer/contractor principle to government-funded research
as an attack on the unity of research and the autonomy of researchers and the Haldane Principle, and
for its recommendation (accepted by Government) to transfer funding for applied research from three of the Research Councils to the control of the corresponding Government Departments.
However, Rothschild’s recommendation of the customer:contractor approach had a more important and lasting adverse effect on the ways in which Government interacted with the science community and consequently on the ability of government to access expert advice for policy-making; I will contend that, in making this a commercial transaction, government scientists took on a lot of contract management work at the expense of analysis and advice, while the research community became more focused on contractual delivery than dialogue with users.
I focus on the operation of Rothschild in two departments, the Ministry of Agriculture Fisheries and Food (MAFF) and to a lesser extent the Department of the Environment (DoE),Footnote2 the two departments which merged as Defra in 2001. This is both because that reflects my direct experience (I worked in MAFF and Defra for much of my career after 1983, and worked closely with colleagues in DoE) and because, in practice, these Departments were the most directly affected by Rothschild in the longer term.Footnote3 It is also the case that it was MAFF’s concerns in the 1960s about the unresponsiveness of the Agriculture Research Council (ARC) to Ministry and farming industry needs that provided the spark that led to the Rothschild fire.
Background—MAFF and the ARC before Rothschild
The history of relationships between MAFF and ARC concerning agricultural R&D is one of mutual suspicion (Williams, 1973; Thirtle et al., 1991; DeJager, 1993). ARC was determined to ensure that scientists had control over the direction of their own research (see for example, Henderson, 1981); MAFF was determined to ensure that what it saw as national needs should lead research funding (Winnifrith, 1962). The position by the 1960s was that control of R&D had passed from MAFF to ARC under the Agriculture Research Act of 1956.
In the run up to that decision, MAFF and its Minister had fought back at the efforts of the research council to take over:
The minister argued that research was still essentially a matter of agricultural improvementFootnote4 for which the Ministry of Agriculture was responsible. He … claimed that the needs of farming had to be the main inspiration of research, and in this regard the Ministry of Agriculture was in a much better position to assess those needs [my emphasis] than the Agricultural Research Council. (cited in DeJager, 1993: 148)
It is notable that the Minister’s argument as reported is in essence the same as that which leads Rothschild to his famous recommendation that “applied R&D must have a customer” (HMSO, 1971; Rothschild report, Paragraph 8).
Intriguingly, though ARC’s principles for the development of agriculture research (Henderson, 1981: 91) were entirely focused on productivity and output, these concerns about the lack of responsiveness of ARC persisted (Ruttan, 1982). Arnon (1989) notes that MAFF tried to overcome this by parallel funding for applied research at the laboratories and experimental farms of its National Agricultural Advisory Services (NAAS)Footnote5 but that there was still poor articulation between “basic” research and applied and developmental research.
In due course, this matter was discussed at high level in Government and a senior official committee (the Osmond Committee, led by the Civil Service Department) was established in 1970 to assess approaches to improving the situation.Footnote6 Their report proposed to transfer ARC to MAFF; the journalist Tom Wilkie, in his critical account of “British Science and Politics since 1945” (Wilkie, 1991: 78) describes this as “an attempt by [MAFF] to hijack the [ARC]”, an overstatement, though it is clear that MAFF made a convincing case about its frustrations. The proposal was the “immediate cause” of Dainton and the Council for Scientific Policy (CSP) starting work (HMSO, 1971; CSP Report, Paragraph 2); being aware of the likely objections from that source, the Government also asked Lord Rothschild to consider the issue.
At this point, it is important to note that the whole of the discussion about the direction and management of agricultural R&D from the 1930s to the 1960s was focused on what we would now think of as research for improving agricultural production (Henderson, 1981; Ruttan, 1982; Arnon, 1989; DeJager, 1993). In that respect, there was no difference in long-term goals on the part of ARC or MAFF, only, and significantly, a different understanding of how this goal was best to be delivered. This point becomes important (1) because it points to an assumption underlying the Rothschild analysis about the purpose of agricultural research, and (2) in the context of subsequent changes to MAFF’s goals; in the later 70s and 80s, it became clear that agricultural overproduction was becoming a problem and new goals emerged to do with environmental protection, conservation and land management in particular.
Rothschild’s conclusions and recommendations
The Osmond Committee discussions can be seen to have led directly to Lord Rothschild, newly appointed head of Edward Heath’s Central Policy Review Staff (the “Think tank”; Blackstone and Plowden, 1988), being asked to undertake an enquiry into Government R&D in 1970.Footnote7 The report, when it came, was, in the words of a Nature (Anon, 1971a) editorial, “Rothschild’s Cat for All Pigeons” and provoked an expected uproar. The New Scientist (Anon, 1971b) called it “The Rothschild Bombshell”.
Rothschild’s two most important conclusions, typically expressed, were:Footnote8
II.6—“the principle that applied R. & D., that is R. & D. with a practical application as its objective, must be done on a customer-contractor basis. The customer says what he wants; the contractor does it (if he can); and the customer pays”.
II.8—“However distinguished, intelligent and practical scientists may be, they cannot be so well qualified to decide what the needs of the nation are, and their priorities, as those responsible for ensuring that those needs are met”.
On the strength of these conclusions, Rothschild recommended the establishment of a customer function for R&D in Departments to be supported by a Chief Scientist (II.10) to help customers (policymakers) decide on R&D programmes and their funding, and approve capital expenditure related to research (II.9).Footnote9 Crucially, he preferred the title of Chief Scientist “to that of Chief Scientific Adviser because he must be closely associated with the decisions in paragraph …. [II]9 … ”, that is, the Chief Scientist’s role was to be primarily that of an adviser on research commissioning.
This is the nearest Rothschild comes to referring to the policy advisory roles of scientists in Government. Indeed, he expressly dismisses it later (III.30) when, referring to a view of the CSP that executive [Government] Departments needed scientific support, he states flatly that they “do not need scientific support but applied R&D, to achieve specific, pre-determined objectives”.
The most controversial new departure was Rothschild’s examination (III.31 and table 4) of the current programmes of the Research Councils and conclusion that NERC, MRC and ARCFootnote10 should be subject to a 50%, 25% and 77% reduction, respectively in their DES vote, with corresponding increases in the votes of relevant departments (DoE, Health and MAFFFootnote11) to enable them to commission applied research. The customer:contractor principle “would have been uncontroversial had Rothschild not sought to apply it to the research councils” (Gummett, 1980: 52); after all, the Ministries of Defence and of Technology were used to this approach in their dealings with industrial research contractors. However, a “military-industrial-scientific complex” of the sort described by Edgerton (2006) was, if one substitutes “government” for “military”, largely absent in the agriculture and health sectors, and not a major component of environmental research; while significant industrial research occurred in, for example, agri-chemicals and pharmaceuticals, little or none of this was funded by the relevant ministries and departments, who focused their funding largely on either their own laboratories or Research Council institutes (Henderson, 1981).
That the move to the customer:contractor principle was made conditional on the establishment by departments of a proper Chief Scientist and customer function, and that a 4-year transition phase should apply during which Departments should commission research from the Research Councils from which the funds were taken, did not, as may be imagined, lessen the outrage caused by this proposal, to which I turn below. In Chapter V, Rothschild dismisses the relevance of the revered Haldane principle, on the basis of a careful reading of what Haldane actually said (HMSO, 1918), especially with respect to “research for General Government use”, as having “little or no bearing on the conduct and management of Government R&D in the 70s”.Footnote12
Rothschild’s other conclusions and recommendations included some brief comments on the coordinating role of the Government’s Chief Scientific Adviser (VI); note that this refers to research coordination, not to coordination of policy advice.
The reaction to Rothschild
Rothschild could be, and was, criticized on many grounds (Ashby et al., 1972; Gummett, 1980: 198ff, 1991) some valid, others less so. I briefly illustrate here some of the main lines of criticism in order to unpack some of the thinking behind Rothschild’s ideas. I then go on to identify what I think is a more important long-term issue that was not covered at the time, concerning the impact of Rothschild’s reforms on the ability of Government departments to access expertise.
The first and most enjoyable set of critiques emerges from the scandal caused by Rothschild’s language seen as, for example, “… terse … to the point of rudeness” (Winner, 1971). The starkness of Rothschild’s language was seen as unseemlyFootnote13 in a Whitehall context though he also complained of being “accused of writing in unintelligible Whitehall jargon” (Rothschild, 1977: 84). Typically, he felt that had he written at more expansive length, “I suspect people would have read it even less carefully” (Rothschild, 1977: 88). He felt he was writing for busy people and much of what he had to say was “commonsense” or “self-evident”, an expression of which he was fond (Rothschild, 1972a). Rothschild himself later regretted his terseness and his choice of words; inter alia, he recognized the “emotive effect” of the choice of “customer” and “contractor”, telling the Select Committee that customer could be replaced by “user” and contract by “commission” (Anon, 1972a). At the time though, no one appears to have commented on how this could put researchers at arms length from their customers and contribute to a defensive “us and them” attitude on the part of researchers.
Another line of criticism is quasi-Burkean; Rothschild was “at odds with tradition” (Winner, 1971Footnote14 ). and flouted the Haldane Principle (for example, Himsworth, 1971). At the time, even Nature dismissed this argument on the grounds that “the Haldane principle has become an archaic survivor from the past” (Anon, 1971d), an interesting opinion in view of the recent resurgence, some 40 years later, in reverence for this archaic Principle (Willetts, 2010).
Many (for example, Sutherland, 1971) challenged the reasoning, or rather absence of reasoning, behind the choice of the appropriate amounts of budget to transfer from the Research Councils to Government departments. This criticism hits home; Rothschild gives no explanation for the figures in his Table 4; it must have been based on some estimate of the proportion of the RC expenditure that was going to applied R&D but, for example, the Zuckerman Committee (HMSO, 1961), only a decade previously, had identified that around 55% of Research Council spending went to applied research, while Rothschild’s figures average out to around 25%. Subsequently, Rothschild (1977: 68–69) fairly off handedly blamed a slip between his draft and the published version of his report for “the important omission” of a tolerance of +/−15% around the 25% figure, without in any way explaining the reasoning behind this figure either. It is noteworthy that even by the time of the Government’s implementing White Paper (HMSO, 1972, Paragraph 50), the figures for the transfers had been reduced by (another unexplained) factor of around 30%. In reality, subsequent statistical summariesFootnote15 of expenditure on R&D show a general if uneven decline in MAFF and DoE, and later Defra spending, especially after 2000, matching a long term if erratic rise in AFRC/BBSRC and NERC expenditure; the Rothschild transfers have long ago been reversed.Footnote16
Outrage at the cuts proposed also led many commentators (a trenchant example is Dixon and Sherwood, 1971) to attack Rothschild on the grounds that, either, he implied that the Research Councils had in some way failed, or that, since they had not failed, there was not a problem that needed fixing. Rothschild (1977: 87) was at pains later to say that he did not think the Research Councils had failed; after all he had been Chairman of one, the ARC between 1948 and 1958, and had drawn attention even then to the lack of connection to farmers of the Agricultural Research Services (Henderson, 1981: 65). Rather, this was an issue about the purposeFootnote17 of and accountability for the expenditure of large amounts of public money and only covered applied research, that is, research undertaken for a specific social/industrial purpose (Calver and Parker, 2015: 7). In any case, according to the editor of Nature (Anon, 1972b) “to chose this point in political history for a plea for stronger control by customers of how public money is spent is likely to warm the cockles of … hearts in the … Government, which means that Lord Rothschild is in a strong tactical position”. And, one has to add, his proposals found political favour and were largely adopted (see below).
More subtly,Calver (2014: Chapter 4) argues that Rothschild’s thinking betrays an old fashioned inductivist understanding of science and contrasts this with the Popperian views emerging in the thinking of the Royal Society at the time, particularly under the influence of Peter Medawar. Medawar, on this view, saw Rothschild as attacking scientific autonomy, which he saw as essential if to the exercise of scientific creativity. Indeed, Rothschild’s very mechanistic account of the customer/contractor principle lends credence to such an interpretation. In practice, Lord Rothschild’s writings provide no indication of his views, if any, on the philosophy of science;Footnote18 his repeated references to “common sense” suggest he might well have made fairly traditional assumptions about science as a purveyor of factual knowledge rather than the more subtle Popperian view of science as tested but provisional understanding. This becomes important when the role of the scientist in policy formulation is considered below. But here it leads to another issue on which Rothschild’s account can be critiqued, also related to the basic/applied distinction.
This is not the place to go into the voluminous literature about the taxonomy of research per se. Footnote19 Suffice to say that Rothschild’s Report appears to be based on a crude, linear model of how science works, which his distinction between “basic” and “applied” science betrays; Rothschild (1977: 85) says clearly that he expects application to be based on the insights of basic research; however, he does go on to point to the ability of applied research to generate issues for basic research and indeed, in an earlier lecture (Rothschild, 1977: 41ff) had expressly referred to the lack of value in making such a distinction, so it is worth examining the origins of his ideas on the management and purpose of research.
Rothschild’s model was very clearly based on his experience as Research Coordinator at Royal Dutch Shell; the address referred to above (Rothschild, 1977: 41ff), which he gave to the Imperial Defence College in 1968 on “Science in Industry”, prefigures many of the features of the Report.Footnote20 Shell’s research commissioning was on a customer/contractor basis (even with its own laboratories), and as Research Coordinator, Rothschild identified the “customer” with the heads of the industrial sections at Shell, not senior scientists, whose role was rather to help and advise on how to address the customer needs. More importantly, here, and in the Report, Rothschild implicitly allows for a dialogue between customer and contractor; the phrase “the contractor does it (if he can)” suggests that there may be some discussion about whether the problem set is soluble by science,Footnote21 which becomes an important element of the role of Chief Scientists. However, the point I wish to bring out here is that (whether or not he can be accused of linear thinking), Rothschild’s ideas were based on an industryFootnote22 model of the value and purpose of applied science, seen as a source of novel products or processes.
Which leads me, finally, to a different critique, not included among those addressed at the time or since, that Rothschild’s understanding of the needs of Government was limited and faulty. Rothschild’s familiarity with the world of business and its research needs, and the nature of his Research Council experience (given the strong post-war focus in the ARC on production; Henderson, 1981), would have led to a thorough understanding of one facet of government research, the promotion of industrial innovation. However, it would not have given him an opportunity to consider the needs of other aspects of policy-making that are less about the future development of novel products and processes, but more focused on analysis of current knowledge to address political problems or on horizon scanning of future social, economic, environmental and technical developments, which might have political significance. In the same address referred to above, Rothschild refers to his having had experience of the Government (Rothschild, 1977: 45) but this turns out to mean his experience in the chair of ARC, not directly with a Government department; his later experience of Government policy-making in the Think Tank was barely 6-months old at the time of the publication of the Report, and as indicated above, the Report was clearly based on prior ideas derived from his time at Shell. Indeed, though he commented on the scientific weakness of departments to the Select Committee (Anon, 1972a), this turns out only to refer to his conception of a Chief Scientist function with respect to R&D commissioning.
Most importantly, Rothschild goes so far as to dismiss the need for proper scientific support within Ministries on typically trenchant terms to the effect that departments do not need strengthening, they need good applied R&D applied to well-specified objectives (Report III:9). In this respect, Rothschild stands charged with grossly underestimating the need for scientific (and other expert) intelligence within policy-making. This is a surprising omission, given that such concerns had been a matter of recent public debate, particularly since the publication of the Fulton Report (HMSO, 1968).
The White Paper of 1972
Green Papers are consultation documents; The Lord Privy Seal’s introduction to A Framework for Government Research and Development (HMSO, 1971) contained what might be thought of as a spoiler; two paragraphs (4 and 5), out of only seven, let the Government’s views on the customer/contractor principle be known; though Paragraph 6 talked about the need for wide debate before conclusion on it, the likely outcome was well-prefigured by the Government’s welcome for the recommendation. And so it came to pass.
The White Paper, with almost the same title, presented to Parliament in July 1972 (HMSO, 1972), only 8 months later, accepted all the key Rothschild innovations, with mostly only minor amendments.Footnote23 By 1974, the transfer of funds had happened; by 1980 much of the hullabaloo had died down; the arguments against could be described as feeble and ill-coordinated (see Gummett, 1980: 229ff). Rothschild’s attacking style, getting in his counter-attack first, had left his opponents wrong footed; even the Editorials in the science journals (for example. Anon, 1972b) had found much of Rothschild’s case, especially for accountability, ungainsayable.
One important difference between the original report and the White Paper lay in how the role of the Government’s CSA and the departmental Chief Scientists is described. The GCSA “advises Ministers on the scientific and technological aspects of Government policies …. He will also be responsible … for advising on the way in which the new arrangements … [for R&D] … are working” (WP Paragraph 10). Correspondingly (WP Paragraph 24), MAFF’s Chief Scientist will “contribute scientific advice to the Ministry on the broad range of policy matters with which it deals. Second, he will have a central role in framing and reviewing the Ministry’s [R&D] programmes …”. Thus the advisory role of Chief Scientists was recognized here, even if not by Rothschild (though the equivalent paragraph on DoE—Paragraph 18—only refers to the customer function). In practice, the exigencies of managing the enlarged budgets were to reduce the time available for other activities.
Rothschild and MAFF and DoE—1974 to 84
MAFFFootnote24 and DoEFootnote25 rapidly appointed Chief Scientists of stature. Charles Pereira was appointed at MAFF (McCulloch, 2008); at DoE, Joe Lyons had been DG Research since its foundation and, in 1976, was followed by Martin Holdgate (Holdgate, 2003). Both were appointed at a very senior level, as Deputy Secretaries, that is, one rung down from the Permanent Secretary and with a board-level voice in departmental affairs. Both settled down to develop relations with their respective research councils (ARCFootnote26 and NERC, respectively). In this, they were supported by newly appointed teams to help the commissioning process; in MAFF, the Chief Scientist’s Group amounted to six very senior scientists (at then SPSO grade) as “Science Liaison Officers” and a complete administrative division to handle the procurement and commissioning process (Henderson, 1981). Similar arrangements applied to DoE’s CSG, though there was more reliance on scientists embedded within policy teams. This constituted a significant increase in scientific capability at head quarters, but it was more or less entirely focused on research commissioning.
MAFF’s relationship with ARC was at that stage relatively cordial (Henderson op cit.). Pereira (McCulloch, op cit.) was a distinguished scientist (FRS) who had been Director of ARC’s East Malling Research Station; his deputy on the agriculture side of his brief was also an ex ARC man, Frank Raymond, formerly of the Grassland Research Institute, and Pereira’s successor in 1977 was Bernard Weitz of the National institute for Research in Dairying at Reading University. Under Pereira’s wing, the relationship developed in a gentlemanly way through discussions with Institute Directors and the formation of “cognate programmes” of basic and applied research, which were still in play by the time the author joined the MAFF Agricultural Chief Scientist’s Group in 1988 under Pereira's later successor, Dr David Shannon, himself another ex ARC Institute Director (Roslin).
MAFF R&D budgets flourished through much of this period, with significant real terms increases, which were shared roughly proportionally between extra and intramural spend. However, the Chief Scientists did not gain control during this phase of the Ministry’s internal R&D spend, the relatively enormous sums being spent through the NAAS (which had its own significant research and field station capability), the Veterinary Service, the Directorate of Fisheries Research or the Food Science Laboratory, let alone the still larger sums spent on non-R&D S&T (monitoring and disease surveillance, inspection, chemical and biological analysis and so on).
DoE’s relations with NERC were initially difficult though settled later (Gummett, 1980: 204), and Holdgate (2003: 284) himself records very bumpy relations with NERC and records the experience of being Chief Scientist as “one of the least rewarding … in my professional life”. DoE’s budgets, though larger than in 1974, showed real terms fluctuations through this period, which cannot have helped. In addition, from even before the Rothschild Report, DoE had spread its contractor base much more widely than MAFF, beyond the Research Council and intramural spend (Everest, 1991).
By the early 80s MAFF was spending over £300m in today’s prices on research and DoE about half that. MAFF and DoE developed quite different models for managing their extramural research budgets; in the case of DoE, the Chief Scientist drew up proposals annually (rather as Rothschild had done at Shell, though this was probably not the explicit model) in consultation with policy colleagues who held the budgets, and had them approved by Ministers. In MAFF, by contrast, the CS, advised by a Joint Consultative Organisation (see Henderson, 1981; this later became the Priorities Board for Agricultural Research), which included industry and Research Council members, decided the budgets, which he held, and their allocations, subject to oversight by a Board R&D sub-committee. On the other hand, from the beginning of the 80s, Holdgate (2003: 225) held not only CS responsibilities, but was also in control of key policy functions (he was in charge of environmental protection policy) and of the strategic Central Planning Unit (not that he saw this as other than a burden); Pereira never had his hands directly on policy levers.
It is notable that throughout this period, and later, the Chief Scientists were organizationally separated from Departmental economists who, in both departments, were embedded in one of the policy DG commands. This separation from the social scientists attracted adverse comment (for example, Spedding, 1984), not least because the purpose of the R&D investment was still seen in essentially economic terms. Arguably, it vitiated the presentation of coherent and comprehensive evidence and advice to policymakers; I often had the impression that policymakers could, on the contrary, play different “experts” off each other.
The 80s through the 90s
By the late 1980s, the Chief Scientist role in both departments was downgraded. In DoE, Holdgate’s successor from 1988, David Fisk continued at Under Secretary level to hold a central policy role as well as the CS function, while in MAFF the DG-level post was split between separate, but lower graded Chief Scientists for Agriculture and for Food and Fisheries when Weitz retired in 1980 (Herman, 1981). Frank Raymond (Anon, 2012) and later Geoffrey Burgess (ex Agriculture Development and Advisory Service) and then David Shannon (from 1986) took over as Agriculture Chief Scientists; a Chief Scientist Food and Fisheries at the same grade matched them, occupied in sequence by George Elton, Tom Crossett, Mike Knowles and Howard Denner; the Fisheries function was moved to the agriculture CS in the 1990s. It was not until MAFF and DoE were merged to form Defra in 2001, that a board level CSA post was re-established.Footnote27
The advent of Mrs. Thatcher’s government brought further significant changes, which led in effect to a deepening and further embedding of Rothschild principles. Agar (2011) interestingly identifies Mrs. Thatcher’s engagement with Rothschild in her role as Secretary of State for Education in Heath’s Government, at a key meeting in 1971, as a critical point in her development as a Thatcherite; Rothschild, supported by Heath, making the case to her for a market mechanism for research, may have changed her view, on science funding for sure but possibly also on wider issues, towards market solutions. From having supported a “Haldane” line presented by her officials, she moved to support Rothschild. “Years later”, she wrote in her autobiography (Thatcher, 1995: 174–175), “I was able to formulate my own answer to the problem, which is that Government should concentrate on funding basic science and leave its application and development to the private sector”; the “answer to the problem”, when it came, became known as the Near Market Cuts.
The “Barnes Cuts”, as the near market cuts applied to MAFF R&D budgets Footnote28 were known, after the civil servant who led the process, removed some 30% of funding by Government Departments over a short period in the late 1980s to early “90s (Hansard, 1988; House of Lords, 1989; Read, 1989); while the policy was based on the ‘near market’ thinking, a clear money saving target was implicit in Barnes’ terms of reference from the start (C.J. Barnes pers. comm.). My personal observation is that much of the cutting proved to have applied to projects not only of merely mundane scientific interest but also of little interest to industry;Footnote29 only a limited amount of this axed research was in practice taken up by the private sector when Government stopped funding, despite the efforts of, for example, the agricultural Levy Boards (Read, 1989). This implies that, pace Rothschild’s principle that Departments should be in a better position than Research Councils to decide on the applied research priorities of the agriculture industry, they had not been very effective at it, and had not noticed the failures of uptake. Much of the cutting fell on in-house research projects but a significant proportion came from Research Council Institute activity, often lineally descended from research being undertaken before the Rothschild reforms, which does not say much in turn for the Research Councils either, as choosers of applied research objectives, vindicating the views of Nature (Anon, 1981)
The Near Market Cuts themselves were linked to another initiative designed to put much Government policy implementation action on a contractor basis, the Next Steps Initiative (Efficiency Unit, 1988). In line with its proposals, “arms length government” became the norm, under the principle that Government could hold contractors to account for agreed programmes of work. Privatizing public sector activities where possible, and where not, turning Government functions over to Next Steps Agencies, was the tool to enable this contractorisation, even if the agencies, being fully owned by Government, could only operate in a quasi-contractual way, with the main sanctions being exercised though the now more tenuous positions of their Chief Executives.
In the case of Government Laboratories, some, such as DoE’s Buildings Research Establishment and MAFF’s Plant Breeding Institute and the Agricultural Development and Advisory Service, could be fairly easily fully privatized but quite a number of labs remained in Government hands because, in practice, they had an important role in providing specialist scientific services (rather than research) in support of policy-making and implementation. For example, MAFF’s Directorate of Fisheries Research was “agencified” as The Centre for Environment, Fisheries and Aquaculture Science (CEFAS); while it was, and remains, the only significant English centre of fisheries research, and an important contributor of marine environmental research, its main functions had to do with the scientific advice that went into fisheries and marine pollution management and the licensing and inspection of, for example, fish farms.
One consequence of this was that the Chief Scientists in MAFF finally came to engage with the whole of the R&D budget, not only extramural but also with the agencies (though not, at this stage, the budget for other non-R&D scientific services provided by the agencies, which accounted for more than half of their budgets and remained under the direct control of policy customers). However, at the same time, the Rothschild principles were applied more tightly and the responsibility for the R&D budgets in MAFF, as had been the case from the beginning in DoE, was given to policy Under-Secretaries, albeit with a double-lock from the Chief Scientists, whose function became more that of advising on research plans and especially on advising on the overall research strategy of the Department. In the early 1980s, Nature (Anon, 1981) could wonder whether MAFF was “Beating a Retreat from Rothschild”, noting that there had been little change in the structure of AFRC programmes since 1972, but by the end of the decade, Shannon could be cited by Arnon (1989) as indicating that customers were more satisfied with the value of AFRC’s work for them, and that MAFF had had a significant influence on how AFRC had applied the Cuts.
Alongside the Near Market Cuts and Next Steps moves, the Cabinet Office and the Treasury, with the help of Department of Trade and Industry (DTI), was sharpening up its guidance on the management of contractors (Cabinet Office, 1989a). The primary impact was to break up the older-style, relatively large and strategic programme Commissions, mainly focused towards individual Institutes, and recast them as much smaller and more narrowly defined objective-driven projects, which might link PIs in several institutes. This combination of changes realised some of the fears of critics of Rothschild from the beginning, that Government departments would not manage research strategically. Certainly, they caused significant difficulties for research institutes when coherent programme units were broken up, and individual PIs found themselves loosing elements of work which made their posts less viable (Read, 1989); Institute Directors lost a significant degree of control as a result.
From personal observation, I would say that a corresponding problem for the commissioning departments was that the move from macro to micro-management substantially increased the cost and effort involved; it brought many more Ministry scientists into the management of commissioning (science staff numbers in MAFF roughly tripled in this period and the administrative resources had to be increased as well). Budgets were high at the time, even after the 30% cuts, and that meant that more effort had to be put into commissioning larger numbers of smaller projects to ensure the money was spent.Footnote30 Less effort proportionally was put into monitoring the use of the funds and even less into analysing the policy relevance of the output and evaluation. In other words, the scientists became commissioning machines and tended to lose their advisory function for lack of available time. Further, because the CSG model kept scientists in a central team, they remained at a distance from the policy-making Divisions, at least in MAFF;Footnote31 once again, DoE’s model, which had most scientists bedded out in policy units had the advantage of bringing them closer to policy, even if much of their effort was still focused on commissioning.
The Government re-affirmed its support for the Rothschild principles in 1987 in its response to a House of Lords Select Committee on Science and Technology Report (HMSO, 1987), and then again in the 1993 White Paper, Realising our Potential (HMSO, 1993). This latter was overseen by William Waldegrave as the Minister then responsible for science; not surprisingly, given Waldegrave’s personal closeness to Rothschild and memories of the emergence of the Report while he was in the CPRS, as well as the general tenor of Government policy at the time, the White Paper specifically re-endorsed Rothschild’s customer: contractor principle, on the basis of advice from the Prime Minister’s Adviser on Efficiency and Effectiveness, Sir Peter Levene and the then Chief Scientific Advisor to Government, Professor, later Sir William, Stewart (Levene and Stewart, 1993).Footnote32 Chapter 5 of the White Paper addresses Civil Department science; this was still very research focused (barely touching on the non-R&D scientific and technical activities of departments) and, like the rest of the White Paper, was largely concerned with wealth creation. It did however, while emphasizing the policy customers’ role in commissioning research, draw attention to the strategic role of the chief scientists and the importance of their networks, and explicitly repeated the 1972 White Paper’s reference to the advisory role of science in policy-making.
Rothschild forgotten?—the late 90s and the new millennium
If the effect of the Rothschild reforms had in practice been to focus Chief Scientists and their staff on research commissioning at the expense of giving scientific advice to policymakers, this all began to change in the wake of the great BSE crisis and the Phillips Inquiry (BSE Inquiry, 2000). Lord Phillips, among other things, gave a substantial amount of advice on how scientists should advise policymakers and, likewise, how policymakers should engage with scientific advice. Some of this was codified at an early stage by the then Chief Scientific Adviser to Government, Sir Robert May, in what became known as the May Guidelines, which have been successively updated by Sir David King (in the wake of Phillips) and Sir John Beddington more recently (the latest version is at Government Office for Science, 2012). The full detail does not need to engage us; the interesting issue from the point of view of this article is that the role of scientist in government becomes refocused towards giving policy advice and away from the commissioning of research, which becomes a secondary and supporting activity.Footnote33 Chief Scientific Advisers once more replace Chief Scientists as the senior scientists in Departments.
With the abolition of MAFF following not only the BSE crisis but also the debacle of the Foot and Mouth outbreak of 2000, and its backing into DoE to become the Department for Environment, Food and Rural Affairs, a single Chief Scientific Adviser, Professor (later Sir) Howard Dalton FRS (Murrell, 2008), was appointed, once more at Board level as a Director General; I was his deputy as Director for Science. Over the subsequent 10 years, first with Howard and then with his successor Professor (later Sir) Bob Watson CMG FRS, we restructured the combined MAFF and DoE science efforts, taking advantage not only of the new regime engendered by the Guidelines but also of the Labour Administration’s support for “Evidence-Based Policy Making” (Smith, 1996; HMSO, 2001). In summary, over the next 10 years, we:
Put responsibility for science investments and for scientists with policy-making teams, to ensure that policy-making was fully evidence-based
Coordinated specialist advice whether scientific, social scientific, statistical or economic, within policy teams under common leadership
Developed a system for coordinating Defra’s total investments in evidence gathering (from advisory committee work through analysis, research and data gathering) through strategic investment planning (the Defra Evidence Investment Strategies, for example, Defra, 2011Footnote34 ), linked to Government Spending Cycles)
Developed Head of Profession arrangements to ensure specialists retained and developed their particular professional skills
Promoted an outward looking approach to networking internal with external expertise in the Universities and institutes
Putting it another way, we reversed Rothschild’s view that all that departments needed was a good well-specified applied research and reasserted the primacy of analysis and advice. In this respect, the dark clouds of departmental R&D budget cuts, especially since 2001, can be thought as bringing with them a slightly golden lining for departmental science staff of greater engagement with the policy process, more time for analysis and advice, and more priority-driven, less expenditure-driven research commissioning.
