Assessment Data Drives Decision-Making: Q & A with Carmen Schools

Carmen Schools of Science and Technology, a network of schools in Wisconsin, encountered this challenge firsthand when they adopted the Illustrative Math curriculum. They had a strong curriculum, but they needed interim assessments to be able to check students’ learning progress in their curriculum and standards throughout the year. Seeking a robust assessment solution that aligned with their curriculum, state standards, and end-of-year testing, Carmen Schools turned to Cognia Interim Assessments.

In a recent interview with Kyra VandeBunte, the Director of Teaching and Learning at Carmen Schools, we discussed the Cognia interim assessments, how they’ve been received in their network, and the impacts thus far. This interview has been edited for clarity and format.

Question: To give some context, could you provide an overview of your specific role in your school and district?

VandeBunte: My title is Senior Director of Teaching and Learning, and in my current capacity, I mainly support our math curriculum, K through 12, and coach our math coaches. I also do some other special projects, like coordinating summer school and network-wide PD, but my primary focus is on math and math instruction.

Question: Before implementing Cognia Interim Assessments, what were the primary challenges that your district was looking to solve, either more generally or specifically related to math and math instruction? 

VandeBunte: Prior to implementing both the Cognia Interim Assessments and adopting the Illustrative Math curriculum, we did not have a shared math curriculum across our two middle schools and our three high schools. We had, generally, a shared scope and sequence and some shared assessments, but all of that was teacher created. I think we were really struggling with making sure that it was the appropriate level of rigor, especially from the assessment standpoint, so we didn’t really have any interim measures of how students were doing towards our end-of-year goals that were reliable and valid.

One of the things that I think has been impactful with the Cognia Assessments is that they are more rigorous than what is within the curriculum.

Adopting the Cognia Interim Assessments gave us a regular checkpoint to know that we had a rigorous assessment that would align with outcomes for the curriculum and outcomes for meeting grade-level standards. And I think we’ve been able to see that that’s been somewhat predictive too, of how kids do on the state assessments. One of the things that I think has been impactful with the Cognia Assessments is that they are more rigorous than what is within the curriculum. They are more formatted as a standardized assessment and more aligned with what a high-stakes test would look like. And so, we know that we’re getting accurate data because it’s not something that’s just teacher-created, but it is created by people who know and write standardized assessment items.

Question: With your current implementation of assessments, is there a process for determining the effectiveness of your assessments? How has that process been in terms of understanding the impact of these assessments in your classrooms?

VandeBunte: We have a data cycle around every interim assessment that sort of involves administering the assessment, then my team looking at assessment results, then leaders looking at assessment results, then teachers looking at assessment results, and finally using that to determine next steps for instruction and ways to lean in. Now that we have three years of data, we are seeing results, which I think is helping us evaluate the effectiveness of our curriculum and assessments and the quality of instruction.

We have seen positive trends for the most part, and when we look at the same assessment year-over-year, students are performing better over time.

We have seen positive trends for the most part, and when we look at the same assessment year-over-year, students are performing better over time. I think the other piece of that, too, is last summer and into this fall, we did some work with Cognia at looking at the correlation between our specifically our eighth-grade results and then aligning them with our state assessment for both eighth-grade and ninth-grade, and looking at if there was a predictive quality between how kids did on the assessment or how kids did on the interims with how they did on the state summative and we found some positive correlation there.

Question: In what ways did the interim assessments align with the educational objectives of your school? Specifically, could you speak more on your district’s math related objectives and how the assessments fit into that?

VandeBunte: We have had a goal over the last three years to see a 10% increase every year in the number of our students scoring proficient on our state assessments, and that comes from wanting to really combat the learning loss that we saw happen during COVID and distance learning. But also, we are a college prep school. We want all our students to have access to college if that’s what they choose. And so, ensuring that as many of our students are proficient in math as possible is foundational to our mission and what we seek to do educationally.

I think adopting the Cognia Interim Assessments gave us a really clear bar of where our kids were on the road to proficiency and gave us information to be able to intervene and reteach where needed and just have a better sense of what was happening in our classrooms on a more regular basis. It’s kind of like the lead measure instead of the lag measure, so we know what’s happening along the way instead of after the fact.

Question: You mentioned before that you have a data cycle where you review assessment data. Did these interim assessments contribute to that process? How did these interim assessments change or contribute to the use of data in your district?

VandeBunte: The interim assessments really were the driver of creating a new data cycle for us. When we first implemented them, we had a full day of staff professional development at the end of each data cycle, which is when teachers are digging into the data from the assessments and planning for what they’re going to do with them.

Because our math assessments were the first interims that we had in the network, that became the driver [of the data cycle] and then has allowed us to introduce more pieces of data and assessment to look at.

And so, we have a number of steps that sort of are all leading up to that but implementing a quarterly interim then gave us the starting point of the data cycle, and then we could determine the other data points that we need to look at during this time. And because our math assessments were the first interims that we had in the network, that became the driver and then has allowed us to introduce more pieces of data and assessment to look at on that same timeline too.

Question: When you implemented the assessments, were there any specific trends revealed by the assessment data that had a significant impact on instructional strategies?

VandeBunte: It’s not broad, sweeping trends, but it is allowing teachers to notice when students score low on a particular standard that might be essential to moving forward into the next unit, so they can incorporate more practice and warm-ups in certain conceptual areas. Or, for example, after kids take their interim assessments in the spring, we’ll have our full staff PD Day, and then the week after that is our last week before kids take their state assessments. So, during this time, we are strategizing around the concepts that we want to hit hard next week before kids take the state assessment that will help them review and prepare.

Question: Based on your experience, how do you envision the role of interim assessments evolving at all in your school? Or maybe, are there areas where you would like interim assessments to do more, or, you know, how do you envision them evolving in use at your school?

VandeBunte: I think one of the things that we’ve continued to play around with is our protocol for how teachers are looking at the data and what we are asking them to dig into. There’s so much data and so many ways to cut it. We’re trying a new method this cycle that is a little bit more simplified and really getting to specifically identified standards to look at more deeply and better understand the particular questions we’re asking students around that standard.

That’s a place that we’d like to keep evolving, giving teachers just what they need to really prioritize and plan based on their kids and their responses.

Teachers are diving into the misconceptions and conceptual pieces that they need to reteach around those standards. Rather than having it be very broad and giving people two hours to fill out a Google Doc template, it’s much more focused. And so, I think that’s a place that we’d like to keep evolving, giving teachers just what they need to really prioritize and plan based on their kids and their responses.

Question: Is there advice that you would offer other school leaders in considering these interim assessments? Or alternatively, what was the biggest lesson you learned that you’d like to share?

VandeBunte: I think our biggest lesson learned was really around  buy-in and giving both teachers and students a clear why beforehand and an explanation of what the assessments are, where they come from, and who Cognia is. That can help build credibility and trust in the assessments. We were able to do that over time, but I think in our first year of implementation, we didn’t lean into that heavily enough, and so we had some distrust and reluctance to implement the assessments at first since there wasn’t an understanding of the level of expertise that went into crafting the assessments and why they are the way they are.

Read more about Carmen Schools’ experience with Cognia Interim Assessments in our recent case study.

Kyra VandeBunte serves as Senior Director of Teaching and Learning at Carmen Schools of Science and Technology in Milwaukee, WI. She is a former Middle School Math teacher, STEM Coach, and High School Principal.