VARC Early Warning Indicator Helps Milwaukee Address High School Dropout rates
JULY 2013
A collaborative effort on early warning systems by researchers at the Value-Added Research Center (VARC) seeks to give the Milwaukee Public Schools (MPS) a better tool to reduce its high school dropout rate, according to a paper recently published in the Journal of Education for Students Placed at Risk (JESPAR).
The paper published in JESPAR, Theory and Application of Early Warning Systems for High School and Beyond, was authored by VARC researchers Bradley Carl, Jed T. Richardson, Emily Cheng, HeeJin Kim, and VARC Director Robert H. Meyer.
VARC is housed at the Wisconsin Center for Education Research (WCER), at the University of Wisconsin-Madison. WCER in general, and VARC in particular, have a long history of working closely with MPS. WCER researchers began working with the district in the mid-1990s around issues of content standards and student assessments, and this work expanded to production of annual value-added estimates of school performance beginning in 2000. Since 2004, a VARC staff member has served as an “embedded researcher” in MPS, splitting time between UW-Madison and the Division of Research and Evaluation at MPS. This arrangement provides VARC with a constant presence in the state’s largest school district, and provides MPS with on-demand capacity in matters of research and program evaluation that is rare even in the largest urban districts. In a 2010 report summarizing early results from the VARC-MPS early warning work, Michael Casserly, Executive Director of the Council of the Great City Schools (an advocacy group for the nation’s largest school districts and the funder of the initial early warning work in MPS), characterized the VARC-MPS research partnership as offering “...one of the best models for district-researcher collaborations in the country.”
Working in conjunction with staff from various departments within MPS (including the Division of Research and Assessment, Student Services, and College/Career Readiness), VARC researchers conducted a detailed analysis of academic, behavioral, and demographic information of the district’s students dating back more than a decade to assemble the data to craft the early warning indicator. Obtaining and reviewing data for quality was a major focus of the early stages of work, as just the academic portion of the work (involving student transcript records) contained more than 4.5 million records.
In the JESPAR paper, the VARC researchers describe in detail how the early warning indicator can be used to identify incoming ninth graders who are likely to struggle in high school, and to provide quicker notice to administrators when these students move from being on-track to off-track to graduate. Equally important is the use of the data to assign students identified as at-risk into research-validated interventions designed to prevent them from dropping out; without this critical piece, there is a very real risk that students will continue to simply be labeled as “at risk” without coherent, well-designed interventions. MPS is addressing the use of early warning results by reporting them within the SAIL (Student Academic Indicators for Learning) system, which is part of the district’s Response to Intervention (RtI) framework.
“This research provides a concrete, useful tool; the result of our long-standing and successful partnership with Milwaukee Public Schools,” VARC researcher and paper co-author HeeJin Kim said. “But it goes further than just providing a product, because we’re seeking to improve the existing predictive methods of student performance, and we want to widen the set of outcomes that early warning systems can and should predict.”
To that end, subsequent versions of the VARC work with MPS will add early warning indicators for students in the middle grades (6–8), with a focus on predicting the probability that students will earn enough credits to advance from ninth to tenth grade. Expanding early warning predictions into the middle grades is a key area of focus, as substantial research indicates that waiting until high school to identify at-risk students is too late in many cases. VARC’s early warning work in MPS will also provide predictions for outcomes beyond high school graduation, such as the likelihood that students will enroll, persist, and graduate from college.
In the long term, future VARC research in Milwaukee utilizing the early warning system will explore which academic and behavioral factors differentiate dropouts from graduates, and whether schools which serve similar student populations have different dropout rates. Also part of future plans is a review of how early warning data can be most effectively presented to educators and school leaders in dashboard-style reporting formats (such as the SAIL system), and how new data being collected by MPS (such as standards-based grades for students) can be used to improve the predictive power of early warning models.