Continuous Improvement and Accountability

Section 4 of 10.

Though continuous improvement processes are not new in education practice, they are relatively new in the state policy arena. In the past, state-level strategies for improving educational outcomes at the school and system levels were driven by federal and state requirements and centered largely on the development of one or more annual “improvement plans” that were used to chart a course of action and investment. Statewide systems of school improvement and support were focused largely on compliance and sanctions based on bald, end-of-year data that provided little evidence of how results were achieved and moreover, were not useful for making decisions at the school and system levels. Plans were typically long documents with a focus on compliance and a goal structure with unrealistically short timelines. To be effective, improvement plans and expectations must be concise, span school years not school months, and set the path. Improvement efforts that have a strong impact on student learning show an understanding of the nuisances of the journey. We must embrace that and move from a focus on the structure of the plan to the capacity to execute the plan.

Embedded, systemic continuous improvement is based on multiple sources of evidence. It focuses on what is happening across the many functions of schools and systems that ultimately affect student outcomes, including teaching and learning, resource allocation, school climate and culture, governance, and management. Information is used to make immediate changes and develop long- and short-term plans that are continually adjusted by the communities within schools and districts.

Accountability for continuous improvement enables all actors and stakeholders to know at all times what affects the culture of the system, the talent it musters, its ability to execute what it wants to accomplish, and the extent to which knowledge is used to strengthen performance (See section: Establishing a State Framework for Continuous Improvement Accountability). This information is rich in context and connects to outcomes across all aspects of what schools do.

The goal of a continuous improvement accountability system is to leverage multiple inputs and processes to achieve desired outcomes. There always will be room for improvement, but the following success criteria can serve as a guide for system development. A system is considered effective when:

• Various processes and components of the system are connected and aligned so that they work together as part of a complex whole in support of a common purpose.
• System improvements are driven by a process of continuous measurement and feedback with a focus on collecting and sharing data that informs and transforms.
• System actors understand and engage each other and the system successfully.
• System outputs are of the desired quality and produced within the desired time frame.

Within schools, districts, and systems, this approach translates to a system that:

1. Acknowledges and adapts to the realities, complexities, and uniqueness of schooling.
2. Employs a systemic approach to actively assess, monitor, and improve at all levels with regard to key education factors and high-quality standards.
3. Ensures a holistic understanding of education quality through myriad reliable data and pieces of information, and uses these data to continuously drive and evaluate improvement actions and support services.
4. Provides transparency and accountability through a valid multi-metric, non-punitive representation of data and information.
5. Identifies, acknowledges, and engages all stakeholders.
6. Provides constant and consistent reinforcement, guidance, and accountability of all stakeholders and factors toward a shared vision.

In a recent book chapter, Jennifer O’Day of the American Institutes of Research and Marshall Smith—a senior fellow at the Carnegie Foundation for the Advancement of Teaching who served as a key advisor at the Department of Education during three administrations— summarizes what the research literature says continuous improvement does and how it differs from recent approaches to accountability. In “Quality and Equity in American Education: Systemic Problems, Systemic Solutions,” the authors write that continuous improvement is focused on system outcomes for a defined population of beneficiaries and the processes that lead to those results. Continuous improvement sees variation in performance (including “failure”) as an opportunity for improvement and learning and takes a system perspective, recognizing that systems are designed to get the results they produce. In other words, if you want to change the results, you have to change the system. Unlike recent policy changes, continuous improvement is based on evidence that includes outcomes, processes, and resources, and embeds measurement into the day-to-day work of the system and its participants. Typically, continuous improvement efforts involve a coherent methodology and processes, such as the Plan-Do-Study-Act (PDSA) cycle used by engineers.[5]

The authors also identify four ways in which CI differs from typical approaches to accountability used by state education systems. Continuous improvement does the following:

1. Focuses on root causes, not just outcomes. Rather than focus exclusively on collecting and analyzing data on student outcomes without information about what happens in the system to produce those outcomes, CI provides detailed information about particular practices to identify important connections between actions and results. By understanding root causes, educators are able to assess the true impact of their actions in order to change what they do to alter the outcomes.
2. Sees failure as a means to improve, not a reason to assign blame or sanctions. Rather than seeing failure as an opportunity for blame and negative consequences, CI uses failure as a means to identify needed assistance and learning. In CI, Smith and O’Day note, “Mistakes and failures are expected; they are both the basis for identifying the focal problem of practice and are opportunities for collective learning about how to make things better.”
3. Enables informed decision-making based on rich context and evidence. Rather than mandate solutions about what should be done when something fails without considering what caused the problem or the strength of the evidence, continuous improvement approaches allow educators to make decisions based on context that enables those within the system to understand which solutions are likely to work for whom and under what conditions.
4. Places the source of accountability and decisions about action for improvement within the system. Rather than placing the source of accountability far from the district and school and not allowing local actors to set goals and identify solutions to problems, the main source of accountability in a continuous improvement approach resides within the system—with key players within the organization focused on the practices and feedback loops they have put in place.[6]