System-wide studies of the for-profit effect on student test scores
In the US, the most noteworthy study of the for-profit effect has been undertaken by Hill and Welsh, who used school-level data to compare for-profit and non-profit charter schools in Michigan. A four year panel of data was constructed (2001-02 to 2004-05) , with all Michigan charter schools which had students taking either the required 4th and/or the 8th grade state level math exam, referred to as MEAP scores (Michigan Educational Assessment program), included in the analysis. A random effects model was employed, controlling for student and district characteristics. The results were published as ‘For-profit versus not-for-profit charter schools: an examination of Michigan test scores’ (Education Economics, 2008), with the authors concluding that they could find no evidence to suggest that the type of ownership of a charter school (profit or not‐for‐profit) affects the delivery of education services either way.
In the same year, Chumacero and Paredes published a study, analysing Chilean voucher reform (‘Should For-Profit Schools Be Banned?’ (MPRA Paper 15099, University of Munich, 2008). In respect of standardised test performance at 4th grade, they showed that pupils in for-profit voucher schools scored 3-15 points higher than their peers in government schools. While non-profit schools performed higher than for-profits on this study, their findings were sufficient to show fears of the profit motive in Chilean education to be unwarranted.
In 2010, Peterson and Chingos’ study of the Philadelphia School Reform Commission’s intervention 2002-08 broke new ground (‘Impact of for-profit and non-profit management on student achievement: the Philadelphia Intervention, 2002-08’, Program on Education Policy and Governance Working Papers Series PEPG 09-02 (Harvard University, 2010). Their research took individual test score data in maths and reading from 2001 and 2002 and then tracked student performance annually to 2008 in order to estimate the relative impacts of the different management frameworks. The study encompassed all 30 elementary and middle schools contracted out to for-profit EMOs, and all 16 contracted out to not-for-profits, in addition to the 71 schools remaining under regular school district management. The impact of not-for-profit management, when compared with regular school district management, was negative in respect of both maths and reading, and more markedly so in maths (albeit statistically significant in only the first year after the intervention began). The impact of for-profit management, on the other hand, was generally positive, though only in maths was it deemed statistically significant. In comparing the relative performance of for-profit and not-for-profit EMOs however, Peterson and Chingos commented as follows:
‘The differential impact of for-profit and non-profit management is especially sizable. Using the estimates given above, students in schools under for-profit management gained between 70 per cent and greater than a full year’s worth of learning in math more each year than they would have had the schools been under non-profit management. All of these differences are statistically significant. In reading, students learned approximately two-thirds of a year more in a for-profit school than they would have had the school been under non-profit management. All but one of the differences are statistically significant’ (p. 4).
Later in the same year, Gabriel Sahlgren provided an even more comprehensive school-level data-set, this time comprising all Swedish schools with at least 15 9th-grade students on roll between 2005 and 2009 (‘Schooling for money: Swedish education reform and the role of the profit motive’ (IEA, 2010). The data-set amounted to 6,935 observations (1,543 schools) and included 725,195 students out of a total of 737,788 graduating in that period, excluding only Special schools and those that do not conform to the standard grading practice.
Sahlgren set out to test the ‘deterioration thesis’ – that is, that the profit motive steadily compromises educational standards over time. Having coded the schools according to ownership structure, straight statistical-profiling showed significant differences in the performance of for-profit, not-for-profit and municipal schools. For-profit independent schools did better than municipal schools and not-for-profit independent schools did better than for-profit schools. In the regressions however, controlling for a wide range of demographic, socio-economic and other contextual factors that influence grades, post-reform for-profit and non-profit independent schools emerged showing more similar positive effects on the average school GPA, raising it by 5.61 points and 6.16 points respectively. (A dummy variable was included to control for the influence of the more exclusive independent schools established prior to the 1992 reforms.)
Applying further controls for (regional) municipality variables, non-profit independent schools raise the average GPA by 5.74 points, whereas the for-profit schools raise it by 4.50 points (p. 18). The impact of the for-profit independent schools was strongest where there were high numbers of pupils from low socio-economic backgrounds, increasing the average school GPA by 11.64 points, compared with non-profits’ 4.39 points (p. 19). For-profit schools were further shown to be beneficial for students from all backgrounds, with the largest effect for students from low-educated families. (Note that this contrasts with the findings of the more widely publicised study of the overall Free School effect by Böhlmark and Lindahl which found the positive effect for pupils with low-educated parents or an immigrant background to be ‘insignificant’ (Böhlmark, A. and Lindahl, M. ‘The Impact of School Choice on Pupil Achievement, Segregation and Costs: Swedish Evidence’, IZA Discussion Paper No. 2786 (Bonn: Institute for the Study of Labor, 2007). Accordingly, Sahlgren concludes, the performance of for-profit independent schools should serve as a guideline for municipal schools regarding minimum acceptable levels of student achievement (p. 20).
In a subsequent study, appended to a later version of the 2010 paper, Sahlgren addresses the problem of endogeneity – that even after controlling for these variables it might still be the case that pupils in Free Schools may be more, or less, able, motivated, etc. than those in municipal schools. Citing a study by Tegle (Tegle, S. ‘Påverkar förekomst av friskolor betygen i grundskolan? – En statistisk analys av samtliga elever i årskurs 9 år 2006’ (Stockholm: Svenskt Näringsliv, 2010), which suggests that not taking this into account would be to significantly underestimate the positive effect of Free Schools generally, Sahlgren explains that he initially decided not to apply controls addressing this phenomenon so as to err on side of caution in testing the deterioration thesis (p. 24). After employing Instrumental-Variable models explicitly designed to deal with endogeneity, the influence of both for-profit and non-profit Free Schools was found to be much stronger, increasing the GPA by 33.74 and 33.86 points respectively and representing an increase of 16.3% in comparison with municipal schools (p. 25).
In summary, not only did Sahlgren’s results overturn the deterioration thesis, but they also strongly suggest that, taking endogeneity into account, for-profit and non-profit schools are equally good at raising standards.
To my knowledge, there have been no further systematic, long-panel evaluations of individual test score gains that estimate relative impacts under similar operating conditions since the publication of Sahlgren’s study.
Revised extract from ‘Profit-making free schools: Unlocking the Potential of England’s Proprietorial Schools Sector’ (© ASI, 2011). James Croft is an IEA education research fellow and Director of the Centre for Market Reform of Education.