One reason why we get bad policies

If the What Works Centre for Local Economic Growth didn’t exist someone would have to invent it. It analyses policies to see which are the most effective in supporting and increasing local economic growth. Although its focus is local, most of its findings have national implications. So far, the centre has looked at a number of policy areas. A theme cutting across all of its findings is that to a large extent we don’t really know what works. Too often, the evidence is inconclusive or lacking.

On access to finance:

• We found very few studies that look at the impact of schemes on both access to finance (direct effect of the scheme) and on the subsequent performance of firms (indirect effects of the scheme).
• While most programmes appear to improve access to finance, there is much weaker evidence that this leads to improved firm performance. This makes it much harder to assess whether access to finance interventions really improve the wider economic outcomes (e.g. productivity, employment) that policymakers care about.
• As with other reviews, we found very few studies that gathered (or had access to) information on scheme costs. As a result, we have very little evidence on the value for money of different interventions.

On business advice:

• There is insufficient evidence to establish the effectiveness of sector specific programmes compared to more general programmes.
• We found no high quality impact evaluations that explicitly look at the outcomes for female-headed or BME businesses.
• We found two high-quality evaluations of programmes aimed at incubating start-ups. Both programmes were targeted at unemployed people and show mixed results overall. However, there is a lack of impact evaluation for Dragons’ Den-type accelerator programmes that aim to launch high-growth businesses and involve competitive entry.

On employment training:

• We have found little evidence which provides robust, consistent insight into the relative value for money of different approaches. Most assessments of ‘cost per outcome’ fail to provide a control group for comparison.
• We found no evidence that would suggest local delivery is more or less effective than national delivery.

As the above suggests, on key areas of government policy we lack evidence of what works, particularly when it comes to determining value for money. Given the billions spent on various schemes this simply isn't good enough.

The way to move forward from this is to work backwards, ensuring that a robust framework of analysis is build into each and every government programme so that we can know how successful (or otherwise) each intervention is. Crucially this should be done in a way that lets it be compared by the same metrics also being measured in other schemes trying to achieve similar outcomes.

Residing in the economic departments of our universities are the brains to do exactly this – to date though, policymakers have lacked the gumption to systematically experiment, measure and evaluate what works. Until they do we will just keep getting ad hoc policies with enough Rumsfeldian known unknowns to make an economist cry.

Philip Salter is director of The Entrepreneurs Network.