Classification Accuracy Update For Grades K-5

Overview

Current SpringMath screening data have been evaluated in recent calls from the National Center for Intensive Intervention (NCII) for inclusion on their Tools’ Chart. Those data can be viewed here https://charts.intensiveintervention.org/screening/tool/?id=5b5f5b465c8db3fd. This document will update existing data, previously reported to NCII, in an effort to continuously evaluate the accuracy of SpringMath screening and decision making.

Comparing these data to the thresholds used by the National Center for Intensive Intervention (NCII; www.intensiveintervention.org) for classification accuracy in MTSS, these data meet the highest standards used by NCII.

How These Data Meet NCII Standards

• Was an appropriate external measure of academic performance used as an outcome?Yes. We used the Arizona Academic Standards Assessment (AASA), which is the year-end accountability measure in Arizona for students in grades 3-5. In K-2, we used a year-end composite score reflecting year end expectations for those grade levels as reported in our past NCII submissions.
• Was risk adequately defined within an RTI approach to screening (i.e., 10th-20th percentile)?Yes, we used below the 20th percentile on the criterion measures, as preferred by NCII. We also report accuracies using proficient/nonproficient on the AASA for grades 3-5, as is generally preferred by systems.
• Were the classification analyses and cut-points adequately performed?Yes. We used the methodology that we have often provided in peer-reviewed published studies.
• The lower bound of the confidence interval around the Area Under the Curve (AUC) estimate ≥ 0.80 andLower bound of AUCs for all grades and all analyses are greater than or equal to 0.80.
• Sensitivity ≥ 0.80 and Specificity ≥ 0.80Sensitivity and specificity are equal to or greater than 0.80 for all grades and all analyses.

About the Sample

These data were collected from a district in the southwestern U.S. for students in Kindergarten through Grade 5. Gender was roughly evenly divided and most participants were either Hispanic or White, each of which accounted for roughly half of 85%-90% of the sample. The remaining 10-15% of the sample self-identified as Black, Multiple, Asian, or American Indian/Alaska Native. Thirteen to eighteen percent of the sample received special education services and 1 to 4% of students were considered English Language Learners. The samples for the fall and winter analyses were very similar and full details are provided for both samples in the following tables.

Sample Details

Sample Details for Fall

* Fall 2024 (screening), spring composite or state assessment 2025 (criterion); Percents may not sum exactly to 100% due to rounding.

Sample Details for Winter

*Winter 2024 (screening), spring composite or state assessment 2025 (criterion); Percents may not sum exactly to 100% due to rounding.

About Classification Accuracy

Previous research has demonstrated reliability of scores obtained on SpringMath measures and demonstrated a lack of bias in the scores. Details of past research studies examining the technical adequacy of SpringMath measures can be viewed here: https://springmath.org/ebook. This update will focus only on classification accuracy data.

In screening, the most critical validity evidence is classification accuracy. It is not possible to attain adequate thresholds of classification accuracy in the absence of well-correlated scores between predictor and criterion, but correlation alone does not guarantee adequate classification accuracy. In classification analysis research, a rule is applied to determine risk from the screening scores. Scores below the rule are considered at-risk (screening-positive) and scores above the rule are considered not at-risk (screening-negative). Those children also have scores on a reference criterion for which a different rule is applied coding those students as scoring nonproficient on the criterion (criterion-positive) or proficient on the criterion (criterion-negative). These coding procedures result in a four-cell contingency table that allows us to characterize the accuracy of decisions based on the screenings, using the standard classification agreement metrics, the most essential of which are sensitivity (the capacity of the screening to detect true positives or students who will score nonproficient on the year-end test) and specificity (the capacity of the screening to detect true negatives or students who will score proficient on the year-end test). There is always a trade-off between sensitivity (avoiding false negative errors by using a more liberal threshold for screening risk) and specificity (avoiding false positives by using a more stringent threshold for screening risk). This trade-off is unavoidable and the Area Under the Curve from the Receiver Operating Curve makes this trade-off apparent while illustrating the value of the screening in separating true positives from true negatives across the full range of possible screening scores.

Sensitivity

Correct positives ÷ (correct positives + false negatives)

Specificity

Correct negatives ÷ (false positives + correct negatives)

Positive predictive power

Correct positives ÷ (correct positives + false positives)

Negative predictive power

Correct negatives ÷ (false negatives + correct negatives)

Pre-test probability or base rate

Index of true positives in your setting

Pre-test odds

Pre-test probability ÷ (1 – pre-test probability)

The criterion used to evaluate screening accuracy is generally year-end state test performance when those data are available. For younger students, the criterion assessment has to be justified. In our technical adequacy data we use a composite score from three to four administered curriculum-based measures in the spring for students in kindergarten through grade 2.

In the sections that follow, we provide updated classification accuracy data for fall screenings and winter screenings in grades K–5 in SpringMath. We also examine the classification accuracy of classwide intervention risk since classwide intervention is the second screening gate in actual SpringMath implementation.

We report accuracy against composite score performance K–2, and on the Arizona Academic Standards Assessment for which we evaluate on two thresholds: scoring below the 20th percentile on the AASA and scoring nonproficient on the AASA.

In all cases, at every grade level, sensitivity and specificity values exceed .80 as does the lower bound of the confidence interval around the Receiver Operating Curve (ROC).

Scatterplots showing screening accuracy and Receiver Operating Curves are appended at the end of this document for the fall screenings (winter screenings were nearly identical).

Fall Screenings

Fall Screenings Grades K–5 with Year-End Composite or State Test Scores

*Criterion in grades K-2 is the year-end composite measure detailed in our NCII reports. Criterion in grades 3-5 is the total mathematics scaled score for the Arizona Academic Standards Assessment (AASA). The criterion reference group is calculated as below the 20th percentile on the AASA in the analysis sample, consistent with the preferred methodology of the National Center for Intensive Intervention (NCII).

Fall Screenings Grades 3–5 with Year-End State Test Proficiency

Educational leaders generally want to predict who is going to score proficient and who is going to score non-proficient on the year-end state test. Thus, on the same sample of grade 3-5 students for whom year-end state test data are available, we also examined screening accuracy in predicting proficiency on the year-end state test (AASA). Accuracies tracked very closely to predicting which students would score below the 20th percentile on the state test. In all grades, sensitivity and specificity exceeded the most rigorous thresholds of 0.80, as did the lower bound of the confidence interval for the Area Under the Curve from the Receiver Operating Curve Analysis.

*The reference criterion in these data was proficient or non-proficient on the year-end state test in Arizona (AASA). We report the accuracies relative to proficiency because typically district leaders want to predict whether students will score proficient or not on the year-end test.

Winter Screenings

Winter Screenings Grades K–5 and Year-End Composite or State Test Scores

* Criterion in grades K-2 is the year-end composite measure detailed in our NCII reports. Criterion in grades 3-5 is the total mathematics scaled score for the Arizona Academic Standards Assessment (AASA). The criterion reference group is calculated as below the 20th percentile on the AASA in the analysis sample, consistent with the preferred methodology of the National Center for Intensive Intervention.

Winter Screenings Grades 3–5 and Year-End State Test Proficiency

Educational leaders generally want to predict who is going to score proficient and who is going to score non-proficient on the year-end state test. Thus, on the same sample of grade 3-5 students for whom year-end state test data are available, we also examined screening accuracy in predicting proficiency on the year-end state test (AASA). Accuracies tracked very closely to predicting which students would score below the 20th percentile on the state test. In all grades, sensitivity and specificity exceeded the most rigorous thresholds of 0.80.

*The reference criterion in these data was proficient or non-proficient on the year-end state test in Arizona (AASA). We report the accuracies relative to proficiency because typically district leaders want to predict whether students will score proficient or not on the year-end test.

Classwide Intervention

Classwide Risk Grades K–5 and Composite or Year-End State Test

In SpringMath, classwide math intervention is used as the second screening gate. Thus, it can and should be evaluated for accuracy as a screening mechanism. We have previously reported accuracy data for classwide math intervention as a screening mechanism on the NCII tools’ chart meeting the criteria to earn full bubbles at some grade levels. Data are presented below for classwide intervention as a screening for all grades K–5. The proportion of opportunities to be at risk that a given student met the risk criterion was the predictor score. The reference criterion used below is scoring below the 20th percentile on the spring composite in Kindergarten through Grade 2 and scoring below the 20th percentile on the year-end state test in Arizona (AASA) in Grades 3–5.

*The criterion for these analyses was scoring below the 20th percentile on the spring composite score in grades K-2 or below the 20th percentile on the AASA in grades 3-5. Classwide risk is the proportion of opportunities to be at risk during classwide intervention for which a student met the risk criterion to advance to intensified instruction.

In all grades, classwide intervention as a screening mechanism exceeded sensitivity and specificity of .80. The lower bound of the Area Under the Curve (AUC) estimates from the ROC analyses also exceeded .80 in every grade.

Classwide Intervention in Grades 3–5 and Year-End State Test Proficiency

Because leaders are typically interested in predicting proficiency, the table below reports the accuracy of classwide intervention risk as a second screening gate in predicting proficiency on the year-end state test in math in Arizona (AASA) for grades 3–5.

*The criterion for these analyses was proficiency on the AASA for mathematics. Classwide risk is the proportion of opportunities to be at risk during classwide intervention for which a student met the risk criterion to advance to intensified instruction.

Decision Accuracies and Receiver Operating Curves for Fall Screenings

Each scatterplot shows every student’s fall screening score against their year-end criterion score. The dashed lines mark the criterion cut and the screening cut, and color shows the decision outcome: correct positives (green), false positives (orange), false negatives (red) and correct negatives (gray). Each ROC curve plots sensitivity against 1 – specificity across every possible screener cut. The faint green band marks the NCII Full Bubble qualifying region (sensitivity ≥ 0.80, false positive rate ≤ 0.20). Winter screenings were nearly identical. Select a chart to enlarge it.

© 2026 SpringMath Accelerate. All rights reserved.