First indications from Peterborough – what do they tell us?

Last week was a big week for Social Finance as reoffending data on the Peterborough pilot was published by the Office of National Statistics. This gives the first early sense of how our first Social Impact Bond is doing. In this blog I want to explore the results a little and some of their implications.

So, first, the numbers, or to give it its full title:

Peterborough (and national equivalent) interim re-conviction figures using a partial (19 month) cohort and a 6 month re-conviction period

 Peterborough

National

Discharge Period

Cohort size

Binary

Frequency

Binary

Frequency

 Sep05-Mar07

725

39.70%

72

36.60%

61

 Sep06-Mar08

870

40.30%

81

37.80%

64

 Sep07-Mar09

1031

40.70%

84

38.30%

68

 Sep08-Mar10

981

41.60%

87

37.30%

69

 Sep10-Mar12

844

39.20%

81

39.30%

79

Binary: Reconviction rate over six months
Frequency: Frequency of reconviction events per 100 offenders within six months

Three topics to cover:

–          Is the Peterborough SIB working?

–          What do these numbers tell us about whether investors are likely to get paid?

–          Do they have any implications for developing the national recidivism PbR work?

1.  Is the Peterborough SIB working?

Put simply, it would appear so. The best way to show this is to index the results so that you can see them together and then to plot Peterborough relative to the national data:

Rebased reoffending data

So, the key measure for us is the one that we will be paid on, the percentage change in frequency of reoffending against a comparison group, in this instance the national cohort.

Peterborough relative to national

On that basis Peterborough has shown a 23% relative decline to the national data. On a sample size of 844 this is likely to be statistically significant, so on reoffending within six months, rather than a year, it appears we are making a difference.

Any caveats? A number. This is on the basis of six month reoffending, not 12 months, so one could argue that the impact of our programme may lessen over time. The comparison group, of wider national reoffending, is not as carefully defined as the comparison group that we have developed in the Peterborough model proper, where the reoffending rates of a matched cohort from the police national computer is used. Given this, the comparator group 16% increase over a two year period is something of an outlier, but it is all we have to go on.

So plenty of caveats. But however much these figures are indicative and however tentative and careful we are being; for a programme in its infancy and on its first cohort, this is a great start.

2. What do we know about whether investors will get paid?

So, two completely different numbers to note here:

a) the 23% relative decline discussed above; and

b) the fact that after this change the frequency and binary metric for Peterborough are now in line with the national average.

In other words what we have achieved so far is to move Peterborough from its historically higher rate of recidivism, to the national average. Through one lens we have done tremendously well. Through another lens Peterborough did (almost) exactly the same as the national average. Which lens will be reflected in the comparison group drawn from the police national computer?

If the prisoners in Peterborough are different and thus reoffend more, then this should be picked up in the comparison group as each individual is matched to one as similar as possible.

If the local environment is different, the prison for example, or the courts system, or the police… Then it is much less clear whether that will be picked up by the comparison group. It could be in part, if prisoners going through Peterborough are relatively local (and about 70% are) then those factors could be picked up to some extent in their criminal history and be matched to prisoners from similar environments. For those that are more transient, for example those coming through from London, such effects are unlikely to be picked up.

Locally there has been speculation for a number of years around why Peterborough’s recidivism rate is higher than the national average, and most of that speculation has focused on prisoner mix. But I don’t believe anybody has any evidence to back that up.

So, this all adds up to probably a greater uncertainty as to whether we will be paid for outcomes than we have that the programme is generating outcomes.

3. Any implications for the development of the national PbR programme?

These numbers probably complicate the development of the national PbR programme in one significant way, they give the impression of an increasing trend in reoffending, while the wider crime stats in terms of amount of reported crime and the British Crime Survey has generally been going down.

The key requirement this creates is that the Ministry of Justice needs to be completely transparent with the data and analysis that it is using to develop the counterfactual data. It simply cannot credibly develop it on its own and then tell people the answer. Regional variation needs to be understood, historical variance needs to be understood, a dialogue is needed to develop an acceptable answer.

Secondly, it increases the potential, in my view, for a proportion of outcomes payment to come from a fixed sized pot that is shared out according to relative performance amongst providers. This will resolve some of the uncertainties in bidders minds and show them that, while they may be taking a risk, there is a defined amount of outcomes payments that will be made if they perform better than some of their peers.

For such a pot to work, there should be a requirement to give a minimum spend on rehabilitation in the bidding process. Open book accounting thereafter can ensure that bidders keep to their promises, but the amount that bidders are willing to invest in reducing reoffending can then be used as part of their assessment. This can be used to counter the issue in the Work Programme – that for profit maximising providers the outcomes payments for harder to reach groups are insufficient to invest in trying to get them back into work.

So, the idea that bidders demonstrate a minimum commitment, is vital to maintaining the programmes credibility – that it is about rehabilitation, as opposed to only being about cost cutting and privatisation.

Would I invest? Part 2 on our arms manufacturing social enterprise

This is the follow up post to this one here.

Thanks for everyone’s input on this – I hope you enjoyed it as much as I enjoyed writing it. People’s answers demonstrated the two basic philosophical approaches to looking at the problem:

a)      A deontological or Kantian ethics approach: that investing in weapons manufacture is intrinsically not social. Therefore neither ethical nor social.

b)      A utilitarian approach: that investing in this weapons manufacturer would overall reduce loss of life and civilian casualties and is therefore a social investment.

In my limited experience when there is an argument between people where one is taking a utilitarian approach and the other a Kantian the best  thing to do is to leave and go to the pub, as they are starting from such different background frameworks they are unlikely to achieve consensus.

A route that combined the two was put forward by Phil Caroe of Allia (thank you Phil), in an ethical investment model: If you believe weapons are unfortunately necessary, and therefore in your universe of acceptable investments, then this investment would score highly on a comparative basis with other potential weapon manufacturer investments. This combines the two by using rules to determine your investment universe and then comparative methods to make a selection. My only concern with this is that it is a passive investment methodology, where your starting point is that you have money to invest and a set of possible investments, which are the best ones?

So where do I come out on this? Well if we apply an active impact investor type model for understanding this investment, one would look at three elements:

a)      Does it have a sensible theory of change or evidence base? In other words do we believe that its actions will lead to the outcome that it is seeking.

b)      Is there a better way to achieve the goals than the way proposed?

c)       What might be the unintended consequences and are they serious enough for us to worry about?

When understanding a) there are a number of attractions. The model has thought through a best angle of intervention, where the pressure points might lie. It has thought through how to be credible to its target audience. It understands the market it is in and the change it is seeking to create. There isn’t any evidence base of course, but this kind of change agent role wouldn’t have an evidence base.

As for b) a much deeper analysis would be required with the team and talking to external experts. Other possibilities might be more grant funded, campaigning or lobbying for example, but we should be able to compare them and then make an assessment rather than allowing our chosen investment methodology to determine our answer.

If I have any concerns, it is around c) unintended consequences… In other words the What Ifs? For example, if they made land mines more acceptable again, would more countries start using them again? One can start getting into quite deep analysis of the value of improving defensive strategies if you aren’t careful. To use a different example, Drones are, some might argue, the ultimate social weapon as they minimise civilian casualties and focus on the bad guys. But their very nature makes them easier to use, more acceptable even. The potential consequences of that ease of use are still playing out. There is quite a good discussion on this on the moral maze, a BBC radio programme. You can download the podcast here (The one thing it does demonstrate is that the last person you want sitting at the other end controlling the drone is Melanie Philips from the Daily Mail).

So I would not be inclined to invest myself. But if someone with real expertise in the area felt they had explored all the other options, and that this was the strategic route to making a difference in this particularly thorny problem then I would be open to them investing and considering it social investment. I don’t think the moral hazard quite puts it out by default in other words.

A few things to highlight from all this.

First, when people try to make ethical decisions they can start from at least two very different starting points. Therefore attempting to reconcile all into a universal answer to impact investment decision making is not going to succeed.

Second, finding answers to complex social issues is not easy. Potential points of leverage and strategies are not straightforward or necessarily comfortable. Many of them transgress traditional red lines for foundations or social investors. They might involve private enterprise. They might be perceived as subsidising government. Someone might make a profit. My fear is that in their struggle to work through what to invest in, organisations start by putting these red lines in place. What this means in practice is that some of the most interesting potential answers will struggle to raise investment.

Taxonomy of Social Enterprise

The difficulty for those thinking of funding or investing in social enterprise is that the concept is wide, some might say baggy; a term that encompasses a range of different organisations with very different needs. This can cause sterile arguments about “what is needed to help social enterprises” where each of the participants is concerned about a different type of organisation.

For a more complete taxonomy of social enterprise, I was delighted to find one on a website called 4lenses. You can access it here. Thank you to Kim Alter and her crew for developing it.

So, let’s consider a quick segmentation of the social enterprise universe, for the purposes of thinking through what types of subsidy might be wanted or required, and therefore help funders choose their area of focus and develop a coherent offer.

a)      Serving disadvantaged populations: social enterprises building business models to bring goods or services to those who cannot afford them as they are presently provided. These may need permanent subsidy, may need subsidy while developing the business model or sufficient scale, or while the actual needs and pricing that is possible in the new market is figured out. Microfinance is a good example of this kind of social enterprise. Initially supported with grant capital, as the risks and potential of the market became clearer and therefore lower a wider variety of capital became available. However as the business model became more commercial so many potential customers started to get left behind again, and regulation started to become necessary as the incentive alignment between providers and beneficiaries became less clear.

b)      Change agents: Many social enterprises are looking to change something, typically behaviour, for the benefit of the service user and wider society. This means there is an element of preaching to the unconverted and as John Kingston formerly of Venturesome puts it “you can’t get the unconverted to pay for the priest”. Will need subsidy during the conversion process, which may be a while if there are always new hordes/ markets to focus on. Government may want to subsidise change agents, if they are seeking a change that is aligned with government’s agenda and spending focus. Otherwise such a subsidy is likely to come from those motivated by the change mission of the organisation.

c)       Supply chain supporters: For example fair trade, where an organisation effectively buys its supplies for more in order to benefit the suppliers or the environment. This may need subsidised availability of investment capital but probably need to be able to make the value of the proposition compelling enough to consumers that they are willing to pay a premium for the product. It is not clear that government necessarily has a role to play in such investment, but in the same way that it is comfortable subsidising charitable giving with a tax subsidy even if it isn’t part of the government’s direct agenda, so investment into fairtrade organisations would have a reasonable case for tax incentives similar to gift aid.

d)      Social ownership structures: mutual, co-operatives or similar employee ownership models may be social enterprises but their services may exist in a normal marketplace (John Lewis for example). For those spinning out of government, they may have issues of being thinly capitalised and therefore needing investment. With all such organisations they struggle raising outside capital so getting scale up without some kind of supportive capital can be difficult. Those in social services therefore should have access to social enterprise investment. Those in more commercial marketplaces, for example an employee owned software company I once worked for, don’t have a particular argument for social investment.

e)      Providers of social services: Not all social service providers are social enterprises, some are from the private sector. So this gets to be a tricky area, what justification would there be for government or other subsidy if others don’t appear to need it. Here the potential value of providing supportive investment is high, but there are issues about how you sort the wheat from the chaff…

This last is in some ways the nub of the problem. What differentiates a commercial company from a social enterprise working in the same space, both competing for government contract? Who needs to differentiate them? Why? I’ve been picking round this topic for a little while. Time to tackle it soon…