About this report

Evidence suggests that the UK performs poorly on infrastructure compared to some other wealthy countries. This Institute for Government work programme explores why UK economic infrastructure policymaking is weak and how it can be improved.

The aim of this initial report is to contribute to a discussion about how best to plan, deliver and evaluate infrastructure, by identifying some of the main flaws in recent and controversial ‘megaprojects’. The Institute is undertaking further work on the key issues raised in this initial report.

Subsequent publications will look in detail at:

Follow our work on infrastructure policy: www.instituteforgovernment.org.uk/infrastructure 

Contents

Summary 2

Introduction 5

The weaknesses in government decision making 10

Introduction

Improving infrastructure policymaking

The UK needs to make major strategic infrastructure decisions over the next few years.1 There is a long-standing need to address many challenges, such as:

1. airport capacity in the south-east of England
2. climate change and energy security requirements, for example, the UK is committed to reducing greenhouse gas emissions by at least 80% in the Climate Change Act 2008
3. maintenance and replacement of existing infrastructure (£8.3bn is due to be spent on roads between 2020 and 2025)2
4. and the pressure of a rising population.3

Well-judged infrastructure investment contributes to a successful modern economy. It can support better economic performance by helping raise productivity and creating jobs.4 But poor investment decisions could lock the economy into inappropriate infrastructure systems for many years, with significant harmful effects on future prosperity.5 Bad investments can result in white elephants – projects that waste public money and fail to deliver the economic benefits a better decision could have brought about. The infrastructure built today shapes the infrastructure it is economical to build tomorrow, because assets fit into a wider system – whether the transport network or the national grid.

There are also frequent political flaws such as failure to secure cross-party agreement, which translates into high political risk for investors, and local community opposition, which often leads to delay.6

Paper overview

The Institute for Government has looked before at problems in the way that governments make decisions about infrastructure, particularly in handling the politics. In previous work we have argued that some poor decisions reflect the lack of institutions that could allow politicians, experts, interest groups and local communities to take part constructively in policymaking.7 We have also looked at how best to enhance citizens’ voices in major policy changes.8

The question of UK airport capacity has been considered many times since the 1968 Roskill Commission, yet progress has been slow and successive governments have postponed the decision on where to give the go-ahead.9 When Theresa May’s Conservative Government approved a third runway at Heathrow in October 2016, it provoked a cabinet split and public criticism, on grounds of noise, environmental impact, and the expense of the particular model chosen, which would pass considerable costs onto airlines and passengers.

An air of uncertainty still hangs over the project; a public consultation is underway, to be followed by a national policy statement (NPS) on aviation and a parliamentary vote. Meanwhile, the only new runways built in recent decades have been at London City and Manchester airports. London airports still rely on runways that have been in place since the middle of the 20th century.10

Problems with the expansion of airport capacity in the south-east of England illustrate the failure to create appropriate institutions and methods of serious engagement with local communities, as well as to compensate them for the costs that large infrastructure projects impose on them.11 The continued controversy (even after the final report of the Airports Commission) also demonstrates the challenges of using long-term forecasting to justify schemes that are costly and difficult to deliver in the short term.

The new nuclear power station at Hinkley Point C looks like a sound decision only if energy security and climate change obligations are given greater weight over cost. 7

And even so, critics argue that other models of nuclear plant or offshore wind would be cheaper.12 The deal guarantees French energy supplier EDF a notably high price, at £92.50 per megawatt hour in 2012 prices. At the moment, wholesale electricity prices are significantly less than that.13

However, it is difficult to ascertain the main factors in the Government’s decision, because it has not been transparent about the evidence that led it to favour Hinkley over other projects, which is a flaw in the process itself. The value-for-money assessment appears perfunctory at just three pages long14 and has little reference to the energy trilemma.15

HS2 has been lambasted as a ‘solution in search of a problem’ since it was announced in 2009.16 First proposed as a way of stimulating jobs, it became a way of saving travellers’ time, then of dealing with excess demand on the West Coast Mainline, then of regenerating the West Midlands. This confusion of purpose has led to suggestions that the choice to build it is as much political vanity as cost-effective investment in regional regeneration.17 Critics have questioned the robustness of the data and forecasting used in the Government’s business case for HS2, as well as the impact on journey times, carbon emissions, homes, communities and habitats. Successive governments have struggled to manage competing interests and, in particular, to foster informed, evidence-based discussion with those who will lose out from its construction.

The Thames Tideway Tunnel is the underground tunnel that is being built to deal with sewer overflow in the Thames. Its case highlights the importance of devoting sufficient time at an early stage to appraising different options. The project, which has now begun construction, has a total cost of £4.2bn and will add an estimated £15–£25 a year to London water bills until 2029/30.18 Critics, including the former Ofwat Director-General, Sir Ian Byatt, say that the cost is unnecessarily high, and that cheaper alternatives were never explored properly.

Decision makers on the Tideway project may have fallen prey to ‘project lock-in’.19 That is, perceived political risks, staff fears of openly expressing disagreement and a decision-making process sceptical of innovation, may have conspired to make decision makers reluctant to turn back after making an early commitment.

In 2014, the former chairman of the Thames Tideway Strategic Study, Professor Chris Binnie, suggested that further research was needed to determine the most cost-effective option for managing sewer overflow.20

The consortium of companies bidding for the project, London and Continental Railways Consortium (LCR), made over-optimistic predictions about passenger growth on the Channel Tunnel in the 2000s. The Conservative Government of the time did not challenge them and subsequent re-estimates by external consultants, Booz Allen Hamilton, were also inaccurate.21 By 2004, the revised central case estimate was below both the 1998 and 2001 ‘low cases’. Once it became clear that the projected revenues were inaccurate, the project did not seem like a good bet to investors whom LCR had assumed would provide debt or equity finance for the project. LCR struggled to secure finance, and the Government was forced to step in and guarantee £3.7 billion that LCR would borrow to fund construction.

The project went ahead on the basis of wider objectives, including regeneration and ‘national prestige’.22 But the most recent evaluation estimates that it currently stands as a net loss to the taxpayer.

The Jubilee Line Extension (JLE), which connected Green Park to Stratford and unlocked the expansion of the Canary Wharf business district, is widely considered a success.23 Although it was initially approved with a benefit-to-cost ratio of 0.9524 (implying that its lifetime benefits would fall short of its costs), according to more recent analyses, the project in fact delivered a return of 1.75.25 However, even in the case of a project widely assumed to be a success, the need for more effective analysis, before and after construction, is clear. As it stands, it is unclear how far the economic benefits (such as the revival of Docklands, which is often attributed to the JLE) are the direct consequence of the project. Wider outcomes such as increased employment in the ‘JLE corridor’ could be attributable to other policies, in particular investment in the East End and the Canary Wharf development.26

In summary, each of these examples demonstrates how process flaws can lead to poor project selection and a contested, lengthy, decision-making process. These problems can lead to further delay and instability in vital investment, increasing the cost of capital for projects. In the following sections, we look at these challenges in more detail and show how overcoming them could lead to more confident and timely project selection, with more strategic coherence at lower cost for the UK’s people and businesses.9

Methodology

We recognise that the third and fourth criteria in particular will mean that this sample is not representative of all infrastructure projects commissioned by governments over this period. This report focuses on large ‘megaprojects’ because these are the hardest to get right and, despite their complexity, account for 29 of the projects in the Government’s current major projects portfolio, with a combined total cost of almost £207bn. There is, however, significant literature, not least the Eddington Transport Study,27 which suggests that smaller schemes are often likely to be more cost effective. The key points in the big vs small debate are summarised in a recent Institute for Government explainer.28

There is no national strategy for infrastructure investment

This is not to say that governments have never articulated wider objectives for infrastructure. There have been a variety of national policy statements29 (for example detailed planning guidelines for projects such as new runway capacity in the south-east of England30), but these only cover project-specific developments.

There have also been national infrastructure plans and pipelines31 published annually since 2010 by the Infrastructure and Projects Authority, previously Infrastructure UK. But crucially, neither the plans nor pipelines have articulated a cross-sectoral vision for infrastructure and how it should fit into national goals. The bulk of the national infrastructure ‘plans’ have been wish-lists of projects, rather than clear objectives and frameworks against which projects could be assessed. There are only objectives for each infrastructure sector (roads, rail, ports etc.) within each plan.

Poor decision-making procedures are a major contributing factor to sustained failure and have led to loss of credibility, high costs and delays11

While the plans and pipelines have helped provide some additional certainty to potential investors, they do not constitute an overall vision of how infrastructure investment should be approached and prioritised. There has never been a UK process similar to Denmark’s strategic analyses, which have had cross-party buy-in outlining infrastructure investment priorities.32

The lack of overarching infrastructure strategy has been repeatedly criticised from a variety of perspectives by academics,33 industry34 and politicians.35 In the absence of a strategy, ministers have not clearly explained the choices and sequencing of UK infrastructure,36 contributing to uncertainty on the part of construction firms and investors,37 and higher costs of capital.38

A national strategy enables ministers to make decisions on a consistent basis and prioritise competing infrastructure projects and options. Without a clear strategy and goals, there is a risk that investment in infrastructure will fail to achieve government objectives or at least won’t achieve them quickly and cost-effectively.39 In the absence of an alternative, a ‘predict and provide’ approach has dominated.40 This focuses on responding to expected market demand to the detriment of other potential objectives, such as reshaping demand.

Despite the strong support for a more strategic, integrated approach, attempts to address this problem over the last 20 years have not been very successful. Earlier cross-governmental bodies did not have either the remit or institutional ‘teeth’ to articulate a holistic strategy. For example, the Commission for Integrated Transport – an arm’s-length body of the Department for the Environment, Transport and the Regions (DETR) – only had a remit covering best practice research and advice on specific issues, such as national road traffic targets and transport safety. Its legal status did not specify any mechanisms for how, if at all, ministers should respond to its recommendations,41 so it couldn’t bring its recommendations to the forefront of the debate. Similarly, the Infrastructure Planning Commission, established in 2009, was in operation for less than three years before its powers were transferred to relevant secretaries of state and the Planning Inspectorate.

The creation of the National Infrastructure Commission (NIC) in 2015,42 which the Institute for Government welcomed at the time,43 has been an extremely positive step. Its National Infrastructure Assessment,44 due to be published in 2018, could help resolve some of the issues associated with the lack of national strategy. However, its role could be strengthened. So far there has been little detail on the format of the final report that the NIC will produce, or how it will hold the government to account for its recommendations in a timely fashion. The government is bound to respond to the NIC’s recommendations within a year, but where it disagrees with a recommendation, there is no requirement to precisely justify why it disagrees.45

The NIC could also be more independent of government. Despite ministers indicating that it would be a non-departmental public body,46 it was established only as an executive agency of the Treasury and Michael Heseltine’s recent dismissal as a commissioner47 illustrated its ultimate subordination to party politics. However, a number of interviewees have argued that the NIC’s closeness to the Treasury allows it to influence government policy more effectively than a fully independent body could.

Case study: High Speed 2

The initial proposal to expand the London to West Midlands/North West England route, which eventually became HS2, was first proposed in a 2008 Department for Transport white paper.48 This was then expanded, with the creation of HS2 Ltd to investigate high-speed possibilities in January 2009;49 and officially became government policy in October 2010.50

HS2 provides an excellent illustration of the challenges of making infrastructure decisions in the absence of a strategy. Since it was first mooted in 2008, HS2 has repeatedly been criticised as a ‘solution looking for a problem’.51 Its objectives have repeatedly seemed to shift. Initially suggested as an employment stimulus, it has subsequently been sold as a way to cut travel time, reduce overcrowding on the West Coast Mainline and regenerate the West Midlands. The lack of clarity over the purpose of HS2 has led to lengthy delays in decision making, as the Government has reworked its analysis and communications strategy, at significant cost, to deal with parliamentary and public opposition. On top of this, critical public and parliamentary stakeholders remain uncertain about whether HS2 represents the most cost-effective solution (not least because there is such disagreement about which problem it is aiming to solve), or good value for money. In a 2011 review of HS2, the House of Commons succinctly summarised the problem:

HS2 has repeatedly come under fire in relation to whether it will meet one of the objectives that has been mentioned most often: ‘to build a stronger, more balanced economy capable of delivering lasting growth and widely shared prosperity’.53 Many have argued that there are far more cost-effective options to meet this objective, such as increasing investment in existing regional connections between northern towns and cities.5413 Policymaking Research and develop initial proposal Early options appraisal Agree need for project. Longlist and shortlist ‘optimum’ options for further analysisEconomic analysis Detailed appraisal of shortlisted options based primarily on economic case (cost-benefit analysis) Project-specific Usually either National Policy Statement and Development Consent Order (DCO) or Parliamentary Hybrid Bill ‘Five-case analysis’ Detailed appraisal of chosen solutions based on all five parts of business case Pre-initiation preparatory phase Outline business case Full business case Decision and project approval Strategic outline case Figure 1: Government approval process for programmes and projects

Government does not devote enough attention to assessing early options

As discussed, clear objectives can enable ministers to prioritise projects and make decisions on a consistent basis. These objectives can be as broad as rebalancing regional productivity rates,55 or as specific as meeting demand in a particular market. Often objectives cover both.

At this point in the decision-making process, several options for meeting objectives are proposed and are subject to early analysis, before a limited set of solutions are taken further.* The Treasury’s Green Book, which provides guidance for public sector bodies on how to appraise proposals before committing funds, calls this the ‘strategic outline case’. Figure 1 illustrates this in more detail.

This ‘front-end’ decision making typically involves less than one-third of the total time spent on project development. However, it has a disproportionate impact on outcomes, as most early ‘shaping actions’** occur during this phase.56 As one interviewee put it, in most cases of government infrastructure investment ‘the big fault line in projects [occurs] as you go from a nebulous policy idea into a specific project […] often a nebulous policy idea has come up with a solution without fully understanding it’.57 In other words, governments move to a preferred solution far too quickly.

This is an important shortcoming. Failure to devote sufficient attention to examining early options can result in better solutions being overlooked.58 This problem is amplified because once solutions are agreed and projects announced, it is rare for previous or newly-emerged options to be reconsidered, even if these options are subsequently shown to better achieve the government’s objectives or provide greater value for money.

Government has recognised that lack of attention to early options appraisal is a problem. The 2010 Coalition Government undertook reforms to improve early options appraisal that focused on increasing the external review of decisions. The reforms gave the Infrastructure and Projects Authority59 an active role in reviewing the early options appraisal and initial business case phases of infrastructure projects,60 and separately brought in external challenge panels of academic and industry experts on select major projects.61 However, external challenge is not present in all projects. An independent Airports Commission recommended Heathrow and there is a challenge panel for HS2, but neither approach was used for Hinkley Point C.

Case study: Thames Tideway

The main problem with the early options analysis was that a mixed solution (using a combination of smaller measures to address storm sewage) was not considered in detail. This is an example of the seeming preference within government for large projects. Options appraisal began positively in 2000 with the analysis framed around the problem – that is, ‘assessing the environmental impact of intermittent discharges of storm sewage’62 – rather than a single solution. This aligned with best practice for decision making,63 which highlights the importance of clearly defining the problem rather than beginning with a preferred solution.

However, the scope of the options considered in the initial study narrowed too quickly. The first publication of the Thames Tideway steering group (2005) concluded that the only viable option was to intercept sewage before it got to the Thames, rejecting a combination of ‘sustainable drainage solutions’* and other options as a solution, without proper analysis. A later, 2014 National Audit Office (NAO) report identified that

2005 was the effective decision-making point, as that was when the study had ruled out all other options aside from interception – although the minister with overall responsibility at Defra did not make an announcement to this effect until March 2007.64 In practice, this meant some kind of tunnel was the only solution under consideration.65 This is problematic as later evidence from Professor Chris Binnie, the chairperson of the original study, suggests that a mixture of sustainable drainage solutions and other measures may have been a more cost-effective solution to the original problem. He changed his mind after his re-analysis suggested the Government had overestimated the number of spills that would occur with sustainable drainage solutions.66

Superficial early-options analysis meant that the decision lost credibility, and led to criticism from key stakeholders. The Environment, Food and Rural Affairs Committee first argued that the Government’s consideration of sustainable drainage solutions had been insufficient in March 2011, when analysing the Government’s national policy statement (NPS) on waste water.* According to the Committee:

This initial criticism snowballed. With reference to the Committee’s conclusion, several London borough councils affected by the tunnel claimed that alternatives to a tunnel had not properly been tested. They commissioned Lord Selborne (who was, at the time, a member of the House of Lords Science and Technology Committee and unaffiliated to the project promoters) to undertake a study of alternative solutions.68 Initiated by Hammersmith and Fulham Council, a coalition of councils sponsored and funded the report on the basis that the proposed tunnel would have ‘a significant detrimental impact on the environment in the borough’.69

1. the Government used faulty assumptions which overstated the number of spills that would occur using sustainable drainage solutions
2. further research was required to confirm the most cost-effective solution
3. on the basis of the available evidence, a mix of sustainable drainage solutions and sewer upgrades appeared to be the most cost-effective solution.70

Later that year the former Director-General of the Water Services Regulation Authority (Ofwat), Sir Ian Byatt, similarly accused ministers of dismissing cheaper and more flexible solutions, noting that these had become more widespread in other countries.71

No reconsideration of alternative options occurred, although improvements in modelling demonstrated cost-escalations in the preferred tunnel option, which doubled in price from £2.1bn to £4.2bn (see Figure 2). According to the NAO’s most recent review, Thames Water did not revisit previously appraised solutions (despite improvements in their ability to estimate the number of sewage overspills that would occur under different solutions) because the Secretary of State had already written asking it to proceed with a full-length tunnel in 2007.72 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1,000 5000 (£m, 2014 prices) Jan-04 May-05 Oct-06 Feb-08 Jul-09 Nov-10 Apr-12 Aug-13 Dec-14 May-16 Oct-15, Defra, Costs and Benefits of the Thames Tideway Tunnel 2015 Update Mar-07, Defra, Regulatory Impact Assessment: Sewage Collection and Treatment for London* Feb-05, Thames Tideway Strategic Study: Working Group Report Nov-11, Defra, Thames Tunnel Costs and Benefits Figure 2: Thames Tideway Tunnel projected costs

1. revisit early options appraisal in light of new information
2. incorporate the most up-to-date data on feasibility, costs and benefits
3. reflect technological developments.73

The Green Book also expresses concerns about over-enthusiastic early shortlisting ‘eliminating the optimal solution before it is given full consideration’.74 And it states that both the outline and full business cases offer opportunities to revisit assumptions and analysis, though it is not clear how often this actually happens. 17

The more ambitious the forecast, the more questionable the model

Until the late 20th century, CBAs were relatively unambitious, particularly in evaluating benefits. They focused on easily quantifiable direct monetary effects, taking less account of wider economic impacts and non-market impacts.75 These limitations were noticed and addressed. A number of reviews suggested that externalities (effects that are not reflected in market prices) and wider economic impacts deserved more attention.76,77 In the 2000s, the Department for Transport began developing tools to estimate those impacts.78

Current Treasury guidance endorses ‘analysis which quantifies in monetary terms as many of the costs and benefits of a proposal as feasible, including items for which the market does not provide a satisfactory measure of economic value’.79 Regarding the time horizon of CBA, it recommends that ‘costs and benefits considered should normally be extended to cover the period of the useful lifetime of the assets encompassed by the options under consideration’.80

The more ambitious the analysis in scope and time horizon, the more likely disruption will take place. The Treasury advises departments to make sensitivity analysis* (a way of expressing the uncertainty around particular assumptions) a ‘prominent part’ of any appraisal. But from the conversations that took place with civil servants as part of this research, it is not clear how far such analysis is fed up the decision-making chain.

In the examples we discuss below, critics have chiefly worried that very uncertain long-term analysis was used to justify schemes which would be costly and difficult to deliver in the short term, at the expense of projects which might pose a lesser implementation challenge and provide more immediate benefit.81

Case study: Heathrow third runway

The analysis that the Airports Commission undertook for Heathrow attempted to forecast flight demand for the next 60 years. It did so using diverse datasets and a highly complex set of models,82 using the Department for Transport’s own modelling tools as a starting point.83

The conclusions of this modelling heavily influenced the Commission’s recommendations. They predicted ‘a faster and more substantial increase in passengers and destinations served at an expanded Heathrow than at Gatwick, particularly in the long-haul market’.84 Overall, they assumed linear national and international growth broadly in line with past trends.

However, the aviation market is notoriously difficult to predict. The economist John Kay highlights that moving ‘from Orville Wright’s first flight in 1903 to the introduction of the jumbo jet took barely 60 years’. The next half-century was less eventful but he notes that ‘while the Roskill commission of the 1960s, which reviewed London airport policy, got traffic growth projections broadly right, it did not forecast that the growth would come from low-cost airlines offering point-to-point services’. Even the 2003 air policy review ‘failed to appreciate how the centre of the world economy was shifting east, and that Dubai would come to be the world’s busiest airport for international passengers’.85 The Airports Commission’s assumption of such little disruption to existing trends is, given recent history and the pace of technological change, risky.

The Commission’s base-case forecasts have already been shown up as inaccurate. It predicted Gatwick would reach 40 million passengers by 2024, but the airport got to 41 million in May 2016.86 The Commission can hardly be faulted for inaccurate predictions. All forecasts are, by their nature, doomed to inaccuracy. However, given the divergence of demand from the Commission’s predictions in less than a year, it looks unlikely that 60-year forecasts will stand up for long.

Case study: Hinkley Point C - on the Somerset coast

Some worry that developments in the energy market even over the past three years have reduced the value of the deal. Figure 3 shows how the Department for Energy and Climate Change’s (DECC, now part of the Department for Business, Energy & Industrial Strategy [BEIS]) own forecasts of wholesale electricity prices for the coming decades have changed since the strike price was agreed in 2013. The unanticipated explosion of the US shale gas market drove prices down, while renewable energy sources have benefitted from subsidies and low operating costs, pushing down the price further. UK energy demand has also not grown with national output and consumption (it has been government policy to decouple them). With the possible exception of the expansion of US shale, these other trends are likely to persist, so even the department’s adjusted 2015 forecasts may be too high (see Figure 3). £0 £10 £20 £30 £40 £50 £60 £70 £80 £90 £100 2013 2015 2017 2019 2021 2023 2025 2027 2029 2031 2033 2035 Figure 3: DECC forecasts of wholesale electricity prices (£/megawatt-hour) 2013 forecast 2015 forecast Hinkley contract price (£92.50 at 2012 prices)

This analysis is not intended to suggest that government should never undertake large, transformative, ‘megaprojects’ with uncertain long-term payoffs. But it is worth noting, in respect of the 2006 Eddington report, that government could be more sceptical of investment in major new grands projets – particularly where the returns, often based on long-term forecasting, are highly speculative.87 There is a large academic literature (including by Atif Ansar and Bent Flyvbjerg88) which argues that large megaprojects are inherently more ‘fragile’, that is, at risk of random events ‘breaking’ them, either in terms of investment value or literally. Therefore smaller, modular projects should generally be favoured.

Sometimes megaprojects will be appropriate. However, the nature of megaprojects (their high cost and lengthy lifecycle) requires a more transparent debate about their risks. Civil servants must clearly communicate the difficulties and uncertainty to ministers, who in turn must be open about the challenges with the public. (This issue is discussed in an Institute for Government explainer89).

Ministers and senior civil servants can fail to understand project risk

Conversations with civil servants during our research support our view that some of this information may be lost by the time advice is presented to ministers in a formal submission, limiting their ability to make considered decisions.21

This problem was particularly evident in the work of the Airports Commission. The Commission initially did, to its credit, examine a range of future scenarios for global aviation. In its interim report, it set these out clearly and was even-handed about the range of possibilities. These included ‘low-cost is king’, in which growth in cheap air travel accelerates and demand pivots from Heathrow to Gatwick.90 However, the final report used only the starting point deemed ‘most likely’ for its analysis and recommendations. Why did this range of scenarios effectively disappear? The final report noted that respondents to the interim report ‘raised concerns about the complexity of the approach and the very broad ranges of results produced, arguing that this made it difficult to draw conclusions’.91 On this basis, the final report took the scenario judged most likely as a ‘starting point’, and this is the scenario that provided headline figures for subsequent ministerial statements.92

This was a questionable choice. Best practice from the private sector, in particular theories of scenario planning derived from Shell, stress the importance of not narrowing the range of scenarios under consideration, even if it makes decision makers uncomfortable.93 As difficult as it is to accommodate risk, outlining a series of plausible futures and asking ‘what will we do if this scenario happens?’ is far more useful than simply asking ‘what will happen?’.94

If ministers and senior civil servants fail to understand this choice, they will struggle to communicate their decision, and risk damaging public trust. In the Airports Commission case, at the very least, ministers should have been clear that its conclusions were not a strategy to ‘preserve flexibility’ but rather a ‘bet on the most probable scenario’95 – which in this case was that ‘future demand is primarily determined by central projections published by sources such as the Office for Budget Responsibility, OECD and IMF’,96 that is, a simple extrapolation from past data that historic trends would continue.*

The NAO highlighted in 2011 that there is insufficient data on the returns which equity investors make for the risks they bear.98 Early PFI contracts, for example, generally did not require investors to disclose their returns after contracts had been let. This raises the possibility that the public sector, and ultimately the taxpayer or the consumer, may be paying more than they should for equity investment99 and privately financed infrastructure. In a 2012 report, the NAO suggests that one reason for this may be that the public sector lacks the expertise to challenge investors’ proposed returns rigorously. This lack of challenge directly relates to an inability to understand risk and insufficient commercial skills more broadly (though government has taken positive steps to build commercial capability).100

The Treasury, the department with responsibility for private finance policy, has historically relied on competition to secure appropriately priced contracts, rather than attempting to assess whether the pricing of equity is optimal. According to the NAO, this has generated a return to equity of around 12–15% in contracts, a proportion of which cannot be explained by risks born by investors.101

If civil servants and ministers do not fully understand project risk, the cost of capital for large infrastructure projects will be excessively high 23

and BEIS claimed to mitigate the risk through a gainshare financial mechanism and a cost ‘discovery and verification process’102 to check EDF’s sums, it is not clear that the high rate of return promised to EDF and CGN is justified, and whether the Government could have negotiated a better deal. The Government’s cursory three-page value-for-money publication does little to assuage these concerns.103 It does not explain how the Government determined that the strike price is appropriate for the construction risk EDF and CGN are taking on.

In stark contrast, the Government appears to have struck a better financing deal on the Thames Tideway Tunnel. By judiciously using a targeted support package, the Government was able to secure capital from institutional investors with limited risk appetites: Dalmore Capital, Allianz Capital partners and Amber Infrastructure. The final cost of capital for Tideway was just below 2.5%, which is remarkably low for a big construction project, and closer to the low returns expected in the regulatory asset base models in the energy transmission and water sectors. This significantly reduced the financing cost of the project, reducing the annual impact on household bills from £70–80 to £20–25.104

For Tideway, high-risk, low-probability events, such as the risk of drilling and flooding the Underground were, sensibly, guaranteed by the Government. Where Bazalgette (the consortium responsible for designing, constructing and financing the tunnel) was not able to secure insurance, the Government provided indemnity for claims up to £2.26bn for damage to construction works.105 In addition to these guarantees, there was a separate competition to provide capital. By accepting and making explicit that there are some risks the private sector are not able to bear, the Government clarified the scope of its implicit support for the project, and successfully reduced the overall cost. Tideway represents a major step forward in the public sector’s understanding and handling of risk in comparison to the wholesale risk transfer of earlier projects. (See High Speed 1 below.)

In general, however, insufficient progress has been made on this issue in the last few years – despite Treasury guidance. The Green Book (which has remained largely unchanged since 2003 though it is currently being redrafted) recommends giving prominent position to different possible scenarios and the sensitivity of analytical assumptions to changing external conditions in business cases.106 This should, if practised, make decision makers aware of the uncertainty they face, but there are questions over how consistently this is applied. In any case it is part of the additional guidance rather than a core requirement.

The problem began with the 1987 Channel Tunnel Act. The Act made government funding for international rail passenger services, the primary purpose of the link, unlawful,107 meaning that construction would not start until the project received a financially viable proposal from private sector bidders. The Transport Secretary at the time, Cecil Parkinson, repeatedly clarified to Parliament that the Government would

not provide money for the project.108 There is a strong case to suggest that the Government of the day did not consider the potential difficulties and uncertainties in obtaining private finance for the project at the outset, despite warnings raised by British Rail during pre-appraisal work.109

The consortium that won the PFI contract in 1996, ‘London and Continental Railways’ (LCR), over-forecast passenger demand. The Government took these forecasts at face value, not challenging their accuracy until the planned financing method failed. This was despite both the complexity and the importance of passenger demand forecasts in determining future revenues. As both the NAO110 and Omega Centre note, it is concerning that ‘no independent assessment of the forecasts were made before the deal between government and LCR was signed in 1996’.111 This proved to be a costly mistake.

The most recent re-evaluation by the NAO in 2012 found that actual passenger numbers between 2007 and 2011 were, on average, one-third of the level that LCR originally forecast in 1995, and two-thirds of the Department for Transport’s independent, revised 1998 forecasts. Failure to understand the inherent risks of ‘basing a business case and financing for a project on demand forecasts where there is only a limited track record’112 led the Government to design a financing mechanism which implicitly assumed the central case demand forecasts, provided by a project bidder, were correct.

When LCR’s optimistic forecasts proved to be a significant demand over-estimation, the PFI contract became unsustainable. LCR could not raise finance from the debt or equity markets, and the Government stepped in to restructure the deal, guaranteeing £3.75bn of LCR’s debt, in 1998. This allowed LCR to raise the money required for construction to proceed.113,114 To avoid breaching the terms of the 1987 Act, the 1998 restructure technically did not involve an increase in government grants, but the deal was fundamentally different to that agreed in 1996. The Government, and by extension the taxpayer, was now exposed to risk that had originally been transferred to the private sector. While the restructured deal was in many respects more robust than the original, the Government would have had greater room for manoeuvre was it not for the 1987 Act.

On both demand forecasts and financing, it is not clear that the Government planned for a situation in which demand was lower than the central case forecast. The NAO note that this is not unique to HS1, and has been a consistent problem: ‘Government[s] have not always taken a realistic view of the likelihood of associated risks materialising’, which for HS1 meant exposure to an ongoing liability when the Government guaranteed debt despite lower-than-expected passenger revenues.115

Stated simply, mistaken beliefs about ease of access to private finance and the accuracy of passenger demand forecasts that were based on an inadequate understanding of uncertainty, damaged the decision-making process. If the Government had developed a range of scenarios for the HS1 project including, for example, difficulties accessing private finance or lower demand forecasts, it would have been able to better communicate the risks to MPs and the public, and plan for foreseeable difficulties.25

Government finds it difficult to make decisions which create ‘concentrated losers’

Almost all infrastructure projects have diffuse benefits shared by a lot of people and localised costs, which fall on a smaller number. These ‘concentrated losers’ bear the brunt of a project’s negative consequences, which can include localised property blight, noise, environmental pollution and landscape impacts. There are, of course, steps government takes to mitigate the worst effects. These include attempts to get local ‘project buy-in’ through engagement,116 and compensation via mechanisms such as compulsory purchases. Despite these mitigating steps, infrastructure often prompts opposition and delay, as local people rarely perceive compensation and consultation to be sufficient.117

Delays are amplified by the UK’s constituency-based electoral system and open planning and judicial system, which give significant voice to local concerns. Governments have often been unwilling to confront the problem of concentrated losers directly. They frequently procrastinate, putting off decisions, commissioning new analysis, and developing political work-arounds.118

Successive governments have made little effort to change how they deal with concentrated losers, in contrast to their efforts on other problems identified in this paper. This is particularly disappointing given the examples of deliberative institutions abroad (the Netherlands Alders Commission, the French Commission Nationale du Débat Public and the Australian Productivity Commission) which appear to have made the decision-making process more constructive and consensual.119

Case studies: Heathrow third runway and HS2

The Roskill Commission was established in 1968 to determine the best site for a new airport for London.120 Its 1971 report recommended an airport at Cublington in Buckinghamshire, but this was rejected by the Government on the basis that it was a political non-starter. Upon receiving the Commission’s findings, Edward Heath, the then prime minister, is reported to have commented that the report ‘seems likely to give us [the Conservative party] a lot of trouble’. The Roskill Commission’s technocratic mandate did not long survive the harsh realities of politics. It should be noted that several people without local interests – including Colin Buchanan, a member of the

Commission, and several MPs – raised questions about the validity of the cost-benefit analysis used to support Cublington.121 However, local opposition was the primary driver behind rejection of Cublington, in the heart of Conservative safe seats, as a solution.

The creation of the Airports Commission in 2012, an independent review established ‘to identify and evaluate how any need for additional capacity should be met in the short, medium and long term whilst maintaining a UK-wide perspective’122 bears some resemblance to the Roskill Commission four decades earlier. Despite the Airports Commission providing a clear recommendation in December 2015 to expand Heathrow, ministers delayed their decision to allow for further study of environmental impacts.123 The delay was almost certainly a method of keeping the Heathrow issue out of the 2016 London Mayoral election, pre-empting a rift between Conservative candidate Zac Goldsmith and then prime minister, David Cameron.124 As the Transport Committee aptly summarised:

Sources: Department for Transport, Air Traffic at UK Airports (Table AVI0101); Institute for Government analysis of key airport expansion decisions 27

The Government did eventually proceed with the Airports Commission’s recommendation to build a third runway at Heathrow. Only a few weeks after Chancellor Philip Hammond claimed that ‘long-term economics, not short-term politics, [will] drive Britain’s vital infrastructure investment’,126 the Prime Minister took the exceptional step of suspending collective cabinet responsibility on Heathrow. This was due to Boris Johnson’s longstanding record of opposition as London Mayor, and both his and Justine Greening’s constituency-based opposition. While unusual, this approach should allow the Government to secure the necessary parliamentary approval for the project. It does, however, highlight how difficult political leaders in Britain continue to find it to make the case for national infrastructure if it adversely affects their, or their ministerial colleagues’, constituencies.

The decision to build a third runway has also been challenged in the courts. Shortly before the Airports Commission reported, Greenpeace and several local authorities brought a legal challenge against the Government127 for failing to exclude Heathrow from consideration on the basis of air pollution, and failing adequately to consult local communities. Although rejected in January 2017,128 further legal challenges will almost certainly arise at a later stage in the approval process.

Dan Durrant, an academic at the Bartlett School of Planning, University College London, argues that the handling of the consultation has only added further fuel to the fire of community scepticism.129 A committee of the House of Lords agrees. The Lords committee on the phase 1 bill concluded that they could not ‘stress enough the importance of effective and timely public engagement, something which, we were told time and again, could be improved upon’.130 The scope of local opposition is extensive. The NAO note that HS2 Ltd have had to process around 2,600 petitions through the hybrid bill process,131 which encompasses only a small part of the overall consultation. There has also been judicial delay, as opponents have resorted to using the legal system. In 2013, a series of challenges were brought against the Government by councils and campaigning groups for failures to comply with environmental regulations, compensation, consultation and other decision-making failures.132

One significant difference to the runway expansion debacle, however, is that few MPs are in a position to delay the project. There are no critical cabinet ministers, equivalent to Boris Johnson in political importance who are unequivocally opposed to the project. And the three main political parties in England (the Conservatives, Labour and the Liberal Democrats) all support the project. The hybrid bill comfortably passed its first stage in the House of Commons by 452 votes to 41, in March 2014.133 Accordingly, there has been less political procrastination on the project than with runway expansion, which illustrates the arbitrary way some projects are, and are not, subject to delay based on local concerns. The question of how to deal with concentrated losers created by infrastructure projects will be looked at in detail in an upcoming Institute for Government paper.

Inadequate evaluation misses the opportunity to improve future projects

This is partially due to ill-defined initial objectives, but also an institutional set-up which prioritises cost, time and scale.* The government already has well-established procedures for learning delivery lessons, established both formally through schemes such as the Major Projects Leadership Academy,134 and informally through programmes such as the Crossrail Learning Legacy.135 But both these programmes focus heavily on traditional ‘on-time, on-budget’ measures of project success. The evaluation process further encourages the prioritisation of simple metrics rather than more complex, wider objectives.**

Better evaluation could increase the accuracy of appraisal2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Source: Institute for Government visualisation of Highways England, 'POPE Meta-analysis 2015', Figure 6-15 ' Margin of error of capital cost estimates by appraisal period' 40 35 30 25 20 15 1050 Average error in ex-ante major project cost forecasting (%) Figure 5: Highways England, average error in cost forecasts by appraisal year

Better evaluation could increase the accuracy of ex-ante appraisal (i.e. forecasting done in advance of a project) by increasing our knowledge of how and why policies have impact, the scale of that impact, and the mechanism by which policies result in desired outcomes. This is particularly important for complex wider effects on productivity, investment, employment and redistribution for which the current evidence base is very much inconclusive.136 Indeed, since Highways England began a structured evaluation programme, their ex-ante appraisals have become more accurate (see Figure 5).

Case studies: High Speed 1 and Jubilee Line Extension

Partly this was a result of an inability, or unwillingness, to set measurable targets for wider economic benefits at the outset. The evaluation was only able to go so far as to state that regeneration benefits were ‘clearly important and formed a major part of the decision to proceed with HS1’.137 In fact, the Department for Transport only commissioned a full evaluation of the economic impact of HS1 following criticism from the Public Accounts Committee in 2012 that the government does not properly evaluate major projects, and therefore does not understand the wider economic impact of transport infrastructure compared with alternative options.

But the inability to measure whether or not a project has met its objectives is also a result of the difficulty in identifying a direct cause-and-effect link between specific infrastructure projects and wider economic outcomes. In the case of the Jubilee Line Extension, multiple post-project evaluations have failed to show that employment and productivity gains (an important part of the early rationale for the project) can be directly attributed to the project itself. This is because the project was interwoven with extensive government investment in East End regeneration, most prominently through the Canary Wharf development. Post-project evaluations struggled to account for the concurrent impact of this regeneration.*,138

These methodological difficulties are not confined to these two evaluations. In a recent review of 2,300 transport studies in the Organisation for Economic Co-operation and Development countries, the What Works Centre on Local Economic Growth** found that only 1% meet the Centre’s minimum standards.139 However, while infrastructure decisions should be made on the basis of the best available evidence, this does not mean leaving it to academics. Relying on academia would almost certainly lead to under-investment, given the stringent standards required to demonstrate causation, and inconclusive evidence on wider effects.

There are few if any uncontroversial truths in public policy and there is not a simple technocratic solution to everything. Tools such as cost-benefit analysis are helpful but they do not fully account for behavioural change, network effects or non-linear effects which accelerate beyond a point of critical mass.140 Ministers must therefore use both judgement and evidence. Where projects have been chosen with reference to wider economic impacts, governments should at least attempt evaluation. If, for example, the government argues that increasing local employment is part of its rationale for Heathrow, it could include the numbers of local out of work benefit claimants as one of the evaluation criteria for the project, alongside the standard metrics we would expect on passenger and flight numbers.

In response to variants of the criticisms outlined above, the Department for Transport has outlined its ambitions to integrate evaluation from the start of projects, and now publishes an annual ‘monitoring and evaluation strategy’ update,141 which details the Department’s progress with most evaluation schemes for major projects. The Department for Transport’s evaluation strategy is typical of most government departments, although the NAO’s most recent review of evaluation in government found that seven departments did not have an evaluation strategy.***,142 The Department for Transport also commissioned work on improving the links between appraisal and evaluation in July 2016,143 which recommended greater alignment between modelling undertaken before and after projects. There are, however, pockets of good practice within departments and arm’s-length bodies, such as Highways England.

Learning from good practice: Highways England

Highways England, a government company, completes a ‘post-opening project evaluation’ (POPE) survey of all road schemes one and five years after completion. This compares pre-project appraisals and post-project evaluations with the aim of reviewing ‘whether it had the benefits and impact predicted’.144 This is then summarised in a biennial meta-report of all scheme evaluations.

Highways England’s evaluation scheme is not perfect. As the What Works Centre for Local Economic Growth notes, its evaluation of wider impacts is flawed because it is based on a before/after comparison and the evaluation relies on basic surveys of local businesses. The Centre therefore recommends using control groups to better demonstrate causation in wider economic impacts, where these are an important part of the strategic or economic case for a project.145 Highways England itself admits that its assessments of broader objectives such as economic development are often ‘anecdotal’.146

However, POPE has had a much greater impact through its standardisation and clear focus than the often post-hoc, badly designed evaluations of other major infrastructure projects. Evaluation of roads now follows a standardised procedure which feeds back into pre-project appraisal in a transparent way. Highways England maintains a list of all the issues raised by POPE, tracks their response to them, and shares best practice with project managers and specialists. The most recent POPE meta-report found evidence of more accurate cost-benefit forecasting since 2004,147 and an increase in the overall value for money achieved by schemes since 2008.148 This suggests that their enhanced evaluation process has improved the way they appraise projects before they start.