Life-cycle benefits of early childhood programs: evidence from an influential early childhood program

Summary

There is a substantial body of evidence showing that early childhood programs can boost the skills of disadvantaged children. Most of this research has evaluated the short-run ‘treatment effects’ of these programs, focusing on outcomes such as cognitive test scores, school readiness, and measures of social behavior. So far, few studies have analyzed longer-term effects such as completed education, adult health, crime, and labor income.

New research aims to bridge this gap, focusing on influential early childhood programs for disadvantaged children in North Carolina. Guided by economic theory, the study shows that it is possible to supplement experimental data with non-experimental data to ‘forecast’ the life-cycle benefits and costs of the programs.

The data are pooled from two virtually identical schemes: the Carolina Abecedarian Project (ABC); and the Carolina Approach to Responsive Education (CARE). These two programs generated numerous positive treatment effects, as the parents of the participants received free childcare, helping to facilitate parental employment as well as participation in adult education.

Additional information (as well as information from structural economic models) helps to guide the ‘monetization’ of the treatment effects, making it possible to extrapolate the measured benefits and costs to the full life cycles of participants. Under a variety of plausible assumptions, the results of this study indicate that the tax-adjusted internal rate of return of the program ranges from 8% to 18.3%, demonstrating clear long-term benefits for program participants in adult life. As a result, the estimates clearly demon-strate the ‘social profitability’ of both the ABC and CARE schemes.

This mode of analysis also provides a template for estimating the life-cycle gains of other similar social experiments (for which there is less than full lifetime follow-up). Supplementing experimental data with non-experimental data enhances the infor-mation available from social experiments and using economic theory and econometric methods to generate empirically concordant forecasts enhances the credibility of the procedure.

Not only does this research provide additional evidence to support the design and implementation of early intervention policies like ABC and CARE, the combined method of experimental and non-experimental data, together with theory-led forward-looking projections, illustrates a new way in which forecasted treatment effects can be used to draw wider conclusions.

Main article

Completed education along with the adult income, health, and crime outcomes for participants in early childhood programs can together provide a detailed measure of the overall long-term effects of such interventions. This article draws on a combination of experimental data and economic theory to chart the projected outcomes of program participants in later life. The authors quantify the multiple lifetime benefits of influential early childhood programs for disadvantaged children in North Carolina, focusing on socio-economic outcomes measured through midlife. The high estimated rates of return for the programs demonstrate their ‘social profitability’.

A substantial body of evidence shows that high-quality early childhood programs boost the skills of disadvantaged children.[1] Most of this research reports short-run treatment effects of these programs on cognitive test scores, school readiness, and measures of early-life social behavior. A few studies analyze longer-term benefits in terms of completed education, adult health, crime, and labor income.[2] Rigorous evidence on their long-term effects on social efficiency is scarce.[3]

Early childhood programs can lead to positive long-term effects on completed education, adult health and labor income

This paper quantifies and aggregates the multiple lifetime benefits of an influential early childhood program targeted to disadvantaged children in North Carolina with outcomes measured through midlife. Guided by economic theory, we supplement experimental data with non-experimental data to forecast the life-cycle benefits and costs of the program. The program has a 13.7% (s.e. 3%) per-annum tax-adjusted internal rate of return and a 7.3 (s.e. 1.84) tax-adjusted benefit/cost ratio. We account for model estimation error and forecasting error in our estimate and conduct extensive sensitivity analyses.

Our paper is a template for synthesizing experimental and non-experimental data using economic theory to estimate the long-run life-cycle benefits of social programs. We go well beyond the mechanical applications of the biostatistical surrogate marker literature used in some recent studies in economics that ignore the difference in sampling distributions of experimental and synthetic cohort data.

We pool data from two virtually identical programs: the Carolina Abecedarian Project (ABC) and the Carolina Approach to Responsive Education (CARE)—henceforth ABC/CARE. Both were evaluated by randomized control trials. Both programs were launched in the 1970s. Participants were followed through their mid 30s. Participants started early in life (8 weeks of life) and were engaged until age 5. The programs generated numerous positive treatment effects.[4] Parents of participants (primarily mothers) received free childcare that facilitated their employment and participation in adult education.

The program is a prototype for many programs planned or in place today. About 19% of all African American children would be eligible for ABC/CARE today.[5] Application of the ABC/CARE program to the disadvantaged populations of today would be an effective, socially efficient policy for promoting social mobility.[6]

Our paper addresses a fundamental problem that arises in evaluating social programs. Few program evaluations have complete life-cycle histories of participants. In our data, the oldest experimental subject is in his/her mid 30s. At issue is determining the life-cycle impact of the program. We construct synthetic cohorts to forecast the full life-cycle benefits and costs of the program using non-experimental data guided by economic theory.[7]

As a byproduct, we also address the problem of aggregating evidence across the multiplicity of treatment effects found in ABC/CARE. We estimate economically interpretable aggregates: internal rates of return and benefit/cost ratios that monetize the large array of benefits and costs generated.

Using non-experimental data and information from theoretical models, forecasts can be made to analyze the long-term effectiveness of early childhood programs

We construct synthetic cohorts from non-experimental samples. The cohorts are chosen to approximate the life cycles of experimentals in their post-experimental sample years. To make this approximation, we formulate and estimate production functions for non-experimental samples that predict program treatment effects within comparable non-experimental samples. We also assess their within-sample forecast accuracy in experimental samples.

Some of the inputs of the estimated production functions that we use to generate treatment effects are changed by treatment. They are measured in both experimental and non-experimental samples. If the production functions mapping inputs to outputs across cohorts are unaffected by treatment—i.e., are “treatment invariant,” or “autonomous” in the sense of Frisch (1938), we can safely use them to forecast treatment effects at older ages beyond the experiment follow-up provided that we accurately forecast the path of future inputs. We test and do not reject treatment invariance by comparing outcomes in experimental samples with those forecasted in non-experimental samples that overlap in age.[8] We forecast experimental treatment effects using our estimated production functions applied to non-experimental data with inputs and outputs. We conduct an extensive sensitivity analyses for our baseline forecasting models including examining, for example, alternative assumptions about cohort effects that might contaminate the use of non-experimental samples. We also compare the outcomes of our approach with a cruder matching procedure that replicates our estimates from our more sophisticated procedure.

As well as providing short-term boosts for school performance and social skills, early childhood programs also benefit participants over the long term

.

Empirical Results

Figure 1 shows the contributions to the present value of major program components. The high rate of return and benefit-cost ratio that we find has multiple sources. Our analysis is a template for estimating the life-cycle gains of social experiments for which there is less than full lifetime follow-up. Supplementing experimental data with non-experimental data enhances the information available from social experiments. Using economic theory and econometric methods to generate empirically concordant forecasts enhances the credibility of our procedure.

Figure 1 – Net Present Value of Main Components of the Pooled (Over Gender) Cost/Benefit Analysis Over the Life Cycle per Program Participant, Treatment vs. Control

Note: This figure displays the life-cycle net present values per program participant of the main components of the cost/benefit analysis of ABC/CARE from birth to forecasted death, discounted to birth at a rate of 3%. By “net” we mean that each component represents the total value for the treatment group minus the total value for the control group. Program costs: the total cost of ABC/CARE, including the welfare cost of taxes to finance it. Total benefits: the benefits for all of the components we consider. Labor income: total individual labor income from age 21 to the retirement of program participants (assumed to be at age 67). Parental labor income: total parental labor income of the parents of the participants from when the participants were ages 1.5 to 21. Crime: the total cost of crime (judicial and victimization costs). To simplify the display, the following components are not shown in the figure: (i) cost of alternative preschool paid by the control-group children’s parents; (ii) the social welfare costs of transfer income from the government; (iii) disability benefits and social security claims; (iv) costs of increased individual and maternal education (including special education and grade retention); (v) total medical public and private costs. Inference is based on non-parametric, one-sided p-values from the empirical bootstrap distribution. Dots indicate point estimates significant at the 10% level.

*QALYs refers to the quality-adjusted life years. Any gain corresponds to better health conditions until forecasted death, with $150,000 (2014 USD) as the base value for a year of life.

Evaluating Ad Hoc Forecasting Methods

The quest for long-run estimates from experiments with short-term follow-up has recently led to application of informal procedures for estimating long-term benefits using short-term measures of childhood test scores Chetty et al. (2011); Kline and Walters (2016). We show that in our samples, these procedures give very misleading estimates of true life cycle program benefits by focusing on earnings, not counting the full array of benefits generated, and relying solely on test scores to predict future earnings. Application of their procedures simply using test scores to predict earnings at age 27 gives a tax-adjusted benefit—cost ratio of .58 compared to our estimate of 7.33.

Our methodology provides a more accurate estimate of the benefits and costs of the ABC/CARE program. We better quantify the effects of the experiment by considering the full array of benefits over the whole life cycle. We also better approximate the uncertainty of our estimates by considering both the sampling error in the experimental and auxiliary samples, the forecast error due to interpolation and extrapolation, and the sensitivity of our estimates to externally specified parameters. We account for the possibility that the effects of inputs on outcomes in the synthetic cohort samples are not comparable to the experimental estimates due to the endogeneity of these inputs in the synthetic cohort samples.

Summary

This paper presents a template for constructing economically interpretable summaries of the multiple treatment effects generated from a randomized evaluation of a high-quality, widely emulated early childhood program with follow-up through the mid 30s. We go beyond the usual practice of reporting batteries of treatment effects. We report the costs and monetize the treatments across numerous domains. We estimate the tax-adjusted internal rate of return and the benefit/cost ratio of the program to assess the social efficiency of the program.

The ‘social profitability’ of early childhood programs such as ABC and CARE lies in their long-term effects, where the benefits far outweigh the costs

We use auxiliary information and the information from structural economic models to guide monetization of treatment effects and to extrapolate the measured benefits and costs to the full life cycles of participants. We account for model estimation and forecast error and conduct extensive sensitivity analyses of our estimates to alternative assumptions and methodologies. Under a variety of plausible assumptions, we estimate that the tax-adjusted internal rate of return of the program ranges from 8% to 18.3% and are all statistically significantly different from zero. Our estimates demonstrate the social profitability of ABC/CARE. We show that estimates from a robust nonparametric matching strategy are close to those from our structural approach. We demonstrate by example the perils of widely used forecast procedures to evaluate the long-run benefits of social programs.

This article summarizes ‘Quantifying the Life-Cycle Benefits of an Influential Early-Childhood Program’ by Jorge Luis Garcia, James Heckman, Duncan Ermini Leaf and María José Prados, published in the Journal of Political Economy in 2020. The summary was written by Jorge Luis Garcia and James Heckman, and their co-authors do not necessarily endorse it.

Jorge Luis Garcia is at Clemson University. James Heckman is at the University of Chicago. Duncan Ermini Leaf and María José Prados are at the University of Southern California.

[1] See, e.g., Cunha, Heckman, Lochner, and Masterov (2006), Almond and Currie (2011) and Elango, García, Heckman, and Hojman (2016) for surveys.

[2] Examples include: Heckman, Moon, Pinto, Savelyev, and Yavitz (2010a), Havnes and Mogstad (2011) and Campbell et al. (2014).

[3] Belfield, Nores, Barnett, and Schweinhart (2006) and Heckman, Moon, Pinto, Savelyev, and Yavitz (2010b) present  life cycle cost-benefit analyses of the Perry Preschool Program. Our approach is more comprehensive in terms of the outcomes analyzed, and in the procedure used to obtain standard errors of estimates of the parameters summarizing social efficiency.

[4] A companion paper, García, Heckman, and Ziff (2018), reports these treatment effects. Participants in ABC/CARE benefit in terms of both cognitive and socio-emotional skills, education, employment and labor income, and risky behavior and health. The parents of participants benefit in terms of labor income and education.

[5] 43% of African American children were eligible at its inception.

[6] García and Heckman (2016) estimate that if ABC/CARE were implemented on the current stock of eligible children, the intra-black gap (black disadvantaged relative to black advantaged) in high-school graduation, years of education, employment and labor income at age 30 for females would be reduced by 110%, 76%, 22%, and 30%, respectively. It would eradicate the intra-black high-school graduation gap, reduce the years of education gap to 0.12 years, reduce the employment gap to 14 percentage points, and reduce the labor income gap to 4,075 USD (2014). For males, the program would eradicate the intra-black high-school graduation gap, reduce the years of education gap to 0.18 years, and reduce the employment gap to 9 percentage points.

[7] Ridder and Moffitt (2007) discuss data combination methods. Our methods are related to the older “surrogate marker” literature in biostatistics (see e.g., Prentice (1989)). However, that literature fails to account for endogeneity of variables in synthetic cohorts compared to the exogeneity of data from experiments. That literature also does not provide testable predictions for validation of its forecasts as we do.

[8] See Hurwicz (1962) for the definition treatment (policy) invariance. We build on the methodology of Heckman, Pinto, and Savelyev (2013), who relate intermediate and long-term outcomes in a mediation analysis. However, they do not construct out-of-sample forecasts as we do in this paper.

Further reading

Almond, D., & Currie, J. (2011). Killing Me Softly: The Fetal Origins Hypothesis. Journal of Economic Perspectives, 25(3), 153-172.

Belfield, C. R., Nores, M., Barnett, W. S., & Schweinhart, L. J. (2006). The High/Scope Perry Preschool Program: Cost-Benefit Analysis Using Data from the Age-40 Followup. Journal of Human Resources, 41(1), 162-190.

Campbell, F. A., Conti, G., Heckman, J. J., Moon, S. H., Pinto, R., Pungello, E. P., & Pan, Y. (2014). Early Childhood Investments Substantially Boost Adult Health. Science, 343(6178), 1478-1485.

Chetty, R., Friedman, J. N., Hilger, N., Saez, E., Schanzenbach, D. W., & Yagan, D. (2011). How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project STAR. Quarterly Journal of Economics, 126(4), 1593-1660.

Cunha, F., Heckman, J. J., Lochner, L. J., & Masterov, D. V. (2006). Interpreting the Evidence on Life Cycle Skill Formation. In E. A. Hanushek & F. Welch (Eds.), Handbook of the Economics of Education (pp. 697-812): North-Holland.

Elango, S., García, J. L., Heckman, J. J., & Hojman, A. (2016). Early Childhood Education. In R. A. Moffitt (Ed.), Economics of Means-Tested Transfer Programs in the United States (Vol. 2, pp. 235-297): University of Chicago Press.

Frisch, R. (1938). Autonomy of Economic Relations: Statistical versus Theoretical Relations in Economic Macrodynamics

García, J. L., & Heckman, J. J. (2016). How Would a National Implementation of Early Childhood Interventions Narrow the Intra-Black and Black-White Outcome Gaps? 

García, J. L., Heckman, J. J., & Ziff, A. L. (2018). Gender Differences in the Benefits of an Influential Early Childhood Program. European Economics Review, 109, 9-22.

Havnes, T., & Mogstad, M. (2011). No Child Left Behind: Subsidized Child Care and Children's Long-Run Outcomes. American Economic Journal: Economic Policy, 3(2), 97-129.

Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P. A., & Yavitz, A. Q. (2010a). Analyzing Social Experiments as Implemented: A Reexamination of the Evidence from the HighScope Perry Preschool Program. Quantitative Economics, 1(1), 1-46.

Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P. A., & Yavitz, A. Q. (2010b). The Rate of Return to the HighScope Perry Preschool Program. Journal of Public Economics, 94(1--2), 114-128.

Heckman, J. J., Pinto, R., & Savelyev, P. A. (2013). Understanding the Mechanisms Through Which an Influential Early Childhood Program Boosted Adult Outcomes. American Economic Review, 103(6), 2052-2086.

Hurwicz, L. (1962). On the Structural Form of Interdependent Systems. In E. Nagel, P. Suppes, & A. Tarski (Eds.), Logic, Methodology and Philosophy of Science (pp. 232-239): Stanford University Press.

Kline, P., & Walters, C. (2016). Evaluating Public Programs with Close Substitutes: The Case of Head Start. Quarterly Journal of Economics, 131(4), 1795-1848.

Prentice, R. L. (1989). Surrogate Endpoints in Clinical Trials: Definition and Operational Criteria. Statistics in Medicine, 8(4), 431-440.

Ridder, G., & Moffitt, R. (2007). The Econometrics of Data Combination. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of Econometrics (Vol. 6B, pp. 5469-5547): Elsevier.