Redefining Measurement of Early Learning

What is “quality” in early childhood programs?

Research has generally identified these elements of quality in early childhood programs (Burchinal, 2018; Doucet, Allen, & Kelly, 2015; Helburn, 1995; Meloy, Gardner, & Darling-Hammond, 2019; Wechsler et al., 2018).

• Instructional support strategies
• Teachers’ education level and ongoing professional learning
• Engaging and developmentally appropriate curricula and classroom experiences
• Assessment to inform instruction and program planning
• Meaningful family engagement
• Administrative practices

Burchinal (2018), Helburn (1995), and others have defined the specific elements of quality as fitting into two groups: structural quality and process quality. Structural quality elements include characteristics of teachers and programs, such as education and training, adult-child ratios and group sizes, staff wages and benefits, leadership and administration, parent involvement, inclusion of children with special needs, and inclusion of home language and culture. Process quality elements are the interactions between teachers and children, characterized by emotional support and intentional teaching (Burchinal, 2018; Hamre et al., 2014). Structural quality elements have been found necessary, but not sufficient, to create high-quality programs (Barnett, 2011; Burchinal, 2018); they indirectly influence the more significant process qualities of programs.

The challenge remains to agree on measures that reliably and consistently identify both high-quality programs across the previously discussed elements and the components on which early childhood programs can focus improvement efforts (Leal, Gamelas, Barros, & Pessanha, 2018).

Traditionally, program evaluation has focused on metrics of program quality, practices, and inputs rather than on child outcomes (National Early Childhood Accountability Task Force, 2013). The most common measures of child outcomes focus on foundational academic skills, executive function, and physical development at a fixed points in time (Elango, Garcia, Heckman, & Hojman, 2015).

What do we actually know about quality?

Several key elements of program practice significantly correlate with child outcomes, according to the research. Intentional teaching that focused on scaffolding higher-order skills in individualized one-on-one interactions and small groups predicted later academic success more successfully than direct instruction focused on rote learning (Barnett, 2011; Barnett, 2013; Burchinal, 2018; Camilli et al., 2010). Evidence-based curricula, supported by aligned training and ongoing support for teachers, was found to have a small but significant effect on children’s literacy skills (Burchinal, 2018). Curricula that target specific skills have been found to have a stronger impact on children’s cognitive and social-emotional gains than more general curricula (Burchinal, 2018; Yoshikawa et al., 2013). However, the curricula must be implemented in a developmentally appropriate approach for these gains to persist over time (Yoshikawa et al., 2013).

What are we missing in current determinations of quality?

Our current measures of quality do not assess the content of what is taught or dimensions of quality such as the effectiveness of curricula and implementation, scaffolded learning, differentiated instruction, or engagement of children and families (Burchinal, 2018). The relatively nascent work in measuring “soft skills” (Lefkowitz, 2018, p. 5), social-emotional learning, and children’s approaches to learning presents additional challenges; there are few valid and reliable measures for these outcomes (Meloy et al., 2019). (National Early Childhood Accountability Task Force, 2013, p. 26); however, using a suite of assessments that does capture a holistic picture of a child tends to be costly and time-intensive.

Recent research efforts have focused first on defining quality and then trying to demonstrate a correlation with child outcomes (Burchinal, 2018; Cannon et al., 2017). Unfortunately, associations have been inconsistent and modest when correlating current measures of quality with children’s outcomes.

Instead, we need to flip the process. Research needs to first examine programs where desired child outcomes are consistently strong, and then examine the practices in those programs to identify correlated indicators of quality. This reverse approach may assist the early childhood field in identifying critical components of quality that have been only minimally studied or not even considered. The field can then focus on helping educators improve those components, rather than continuing to measure and rate elements of quality that do not strongly correlate with child outcomes.

Recommendations

Blending academic, theoretical research with practitioner expertise and engagement can lead to a shared understanding of desired child outcomes that are associated with children’s long-term academic, career, and life success. Those outcomes can be correlated to a common set of indicators observed in high-quality programs. These correlations would enable early childhood programs to become more consistent in accurately assessing meaningful elements of quality and identifying specific actions for continuous improvement.

Specifically, professionals in the field could aim to accomplish these objectives.

• Define a common set of desired child outcomes that include short-term success for school readiness, sustained academic and social gains, and long-term success in career and life.
• Invest in the development and validation of developmentally appropriate, holistic measures of child outcomes, with a special emphasis on measures that address gaps in current research.
• Undertake well-designed, experimental studies to identify the specific combination(s) of quality elements that result in positive impacts on child outcomes, resulting in a common set of quality indicators linked to outcomes.
• Review existing measures and develop and validate additional measures of program quality to reflect the common set of indicators that result in child outcomes.
• Implement rigorous longitudinal studies to establish links between high-quality programs and short- and long-term positive impacts on child outcomes.

Conclusion

Redefining the measurement of early childhood program quality by focusing first on desired outcomes, then identifying correlated indicators, has the potential to provide a new approach to identifying and defining high-quality programs. Consistent, reliable, and valid measures can be used across contexts, funding sources, and states or governing authorities to systemically impact early-childhood education. Currently, regulations, requirements, and expectations for quality improvement vary widely under multiple influences and agencies. The proposed research and policy approach would contribute to an aligned system with common pillars of support to guide program quality, resulting in desired child outcomes.

Streamlining funding, research, and data collection through the lens of a common set of quality indicators linked to children’s outcomes, and supported by aligned measurement tools, will allow programs to implement high-quality elements and more effectively ensure children’s success. Ultimately, the entire system will benefit from higher-quality early childhood options that are available to all families and children. Every child should have the opportunity to enter later school experiences positioned for academic, career, and life success.

References

Barnett, W.S. (2011). Preschool education as an educational reform: Issues of effectiveness and access. New Brunswick, NJ: National Institute for Early Education Research.

Barnett, W.S. (2013). Expanding access to quality pre-k is sound public policy. New Brunswick, NJ: National Institute for Early Education Research.

Barnett, W.S., Jung, K., Youn, M., & Frede, E.C. (2013). Abbott preschool program longitudinal effects study: Fifth grade follow-up. New Brunswick, NJ: National Institute for Early Education Research.

Burchinal, M. (2018). Measuring early care and education quality. Child Development Perspectives, 12(1), 3-9. doi: 10.1111/cdep.12260

Camilli, G., Vargas, S., Ryan, S., & Barnett, W.S. (2010). Meta-analysis of the effects of early education interventions on cognitive and social development. Teachers College Record, 112(3), 579-620.

Cannon, J.S., Kilburn, M.R., Karoly, L.A., Mattox, T., Muchow, A.N., & Buenaventura, M. (2017). Investing early: Taking stock of outcomes and economic returns from early childhood programs. Santa Monica, CA: RAND Corporation.

Doucet, F., Allen, L. (Ed.), & Kelly, B.B. (Ed.) (2015). Transforming the workforce for children birth through age 8: A unifying foundation. Washington, D.C.: The National Academies Press.

Elango, S., Garcia, J.L., Heckman, J.J., & Hojman, A. (2015). Early childhood education. NBER Working Paper No. 21766. Cambridge, MA: National Bureau of Economic Research.

Hamre, B., Hatfield, B., Pianta, R., & Jamil, F. (2014). Evidence for general and domain‐specific elements of teacher–child interactions: Associations with preschool children’s development. Child Development, 85(3), 1257-1274.

Helburn, S.W. (1995). Cost, quality and child outcomes in child care centers. Technical report, public report and executive summary. Denver, CO: University of Colorado at Denver. (ERIC Document Reproduction Service No. ED 386297).

Institute of Medicine and National Research Council. (2012). From neurons to neighborhoods: An Update: Workshop summary. Washington, DC: The National Academies Press.

Leal, T., Gamelas, A.M., Barros, S., & Pessanha, M. (2018). Quality of early childhood education environments: Discussion on the concept of quality and future perspectives. In: A. Gregoriadi, V. Grammatikopoulos, & E. Zachopoulou (Eds.), Professional Development and Quality in Early Childhood Education (pp. 29-58). Cham, Switzerland: Palgrave Macmillan.

Lefkowitz, R. (2018). 2018 workplace learning report: The rise and responsibility of talent development in the new labor market. Sunnyvale, CA: LinkedIn Learning.

Meloy, B., Gardner, M., & Darling-Hammond, L. (2019). Untangling the evidence on preschool effectiveness: Insights for policymakers. Palo Alto, CA: Learning Policy Institute.

National Early Childhood Accountability Task Force. (2013). Taking stock: Assessing and improving early childhood learning and program quality. Washington, DC: Foundation for Child Development, The Pew Charitable Trusts, The Joyce Foundation.

Rolnick, A. & Grunewald, R. (2003). Early childhood development: Economic development with a high public return. The Region, 17(4), 6-12.

Wechsler, M.E., Kirp, D.L., Ali, T.T., Gardner, M., Maier, A., Melnick, H., & Shields, P.M. (2018). On the road to high-quality early learning: Changing children’s lives. New York, NY: Teachers College Press.

Yoshikawa, H., Weiland, C., Brooks-Gunn, J., Burchinal,, M.R., Espinoza, L.M., Gormley, W.T., Ludwig, J., Magnuson, K.A., Phillips, D., Zaslow, M. (2013). Investing in our future: The evidence base on preschool education. Washington, DC: Society for Research in Child Development, Foundation for Child Development.

This article is based on King, Holly (2019) The Redefining the Measurement of Early Childhood Program Quality and Child Outcomes [White paper].

Holly King, Ph.D.,  is senior vice president of Evaluation Services for Cognia, overseeing all aspects of accreditation and certification services offered to PK-12 schools around the world. Over her career, Dr. King has directed education programs in private, non-profit, and school district settings.  She has held adjunct faculty positions in Early Childhood Education for eight years at three colleges, teaching adult learners in online and face-to-face settings, and is an adjunct faculty member for the Master’s in Organizational Leadership program at the University of Denver. She also serves as Affiliate Research Faculty for Antioch University's Ph.D. in Leadership and Change in quantitative methods. To her leadership roles, Dr. King brings expertise in best practices in early learning and K-6 education; strategic planning; leadership development; quality assurance; data analysis; health, mental health, and disabilities services; family support; and professional development and coaching. Dr. King holds a Ph.D. in Leadership and Change from Antioch University, as well as master degrees in Early Childhood Education and Leadership and Change.