Introduction

Step 1: Add filters to your columns

Step 2: Determining the numbers that go into cells A and B

Step 3: Determining the numbers that go into cells C and D

Conclusion


Introduction

An Interim Assessment’s Ability to Predict Student Outcomes on the Iowa Assessments

 

Before educators give students any assessment, cut scores for mastery or proficiency need to be established. With interim assessments, such as Skills Iowa, aReading, or CBM-R or any other interim assessment, mastery cut scores are established that determine if students have mastered the material typically taught over a one to three month period. Thus, quality interim assessments are used to determine if students’ learning is on track. With summative assessments, such as the Iowa Assessments, predetermined cut scores categorize students as proficient at the end of a time period during the school year or not, based on their achieved scores. Thus, with any given assessment two big outcomes occur. A student either demonstrates mastery/proficiency or does not.

 

When we combine the possible student outcomes from an interim and a summative assessment, four possible outcomes can occur. These outcomes are listed in the table below. We can use this information to our advantage to confirm that our interim assessments are of good quality and do what interim assessments are designed to do. That is to inform us sooner rather than later which students need additional instruction, outside of core instruction, and if these instructional sessions are making a difference with student learning as measured on the next interim assessment. The assumption is that when multiple quality interim assessments are used across the school year to measure how students are progressing, educators then have multiple sources of data to help them determine the effectiveness of core, targeted and intensive instruction on students’ learning.

Interim Assessment: Skills Iowa Outcome Measure: Iowa Assessments
Fail: Outcomes Assessment Pass: Outcomes Assessment
At Risk (Fail) A B
Not At Risk (Pass) C D

Of course this assumption only holds true if the interim assessment is of good quality—that is, it’s validity, reliability, alignment to the Iowa Core and able to predict how students will perform on the outcome measure with some level of certainty. In this example, we will focus on how to measure the ability of an interim assessment to predict the outcome measure.

 

An initial way to understand and show the usefulness of an interim assessment, in an Excel spreadsheet, would be to color code student scores with a green background which would indicate that a student had demonstrated mastery of the learning target, or a red background which would indicate that the student had not. Color coding interim assessment scores gives us an easy visual that displays how well students performed on interim assessments across the school year. Having all this data on an Excel spreadsheet provides educators with a constant view of how their students performed on interim assessments throughout the school year, and more importantly the impact of instruction. This way of displaying student data can serve as a means of monitoring students’ scores on various interim assessments and their ability to predict students’ outcomes on the summative assessment. However, as educators we have a moral imperative to take a deeper look at this data, by analyzing the predictive ability of interim assessments and make adjustments, if necessary.

 

This deeper look involves calculating the negative and positive predictive power and the overall classification index of an interim assessment relative to the outcomes measure, the Iowa Assessments. These measures can help educators determine the quality of “off the shelf interim assessments,” as well as the quality of teacher developed interim assessments.

 

When using interim and summative assessment scores for predicting purposes, we are not limited to using data from the same year.  The best predictor of this year’s summative assessment scores is last year’s summative assessment scores.  We can use last year’s summative scores as a means to determine the rigor of our interim assessments.  For example, we can use scores from the current interim assessments, with last year’s summative assessment scores, to calculate the negative and positive predictive power and overall classification index.  Use the definitions provided on the Excel spreadsheet to gain a better understanding of the above terms as well as the numbers that go into cells A, B, C and D.

 

Now that you have a clearer understanding of these terms, let’s determine how we can easily calculate these measures using the power of Excel. In doing so you should get an idea of what negative and positive predictive power and the overall classification index measure, and the quality of the interim assessments in being able to predict students’ summative outcomes. These can also be calculated at the end of the school year with current summative assessment scores.

 

Take a minute to review the attributes of cells A, B, C and D in the above table. The simplest explanation of how we are going to determine the numbers that go into each cell is to use the “Sort/Filter” and “number filters” tools in Excel.  We will first filter the data to view only those students who did not achieve mastery on the interim assessment. We will then apply a second filter to sort the Iowa Assessment scores from “smallest to largest.” Then we will have Excel count the number of students who did not pass the Iowa Assessments and put that number in cell A. Then, we will have Excel count the number of students who passed the Iowa Assessments and put that number in cell B.

 

Next, we will undo the original filters by applying a new number filter that will provide a view of students who achieved mastery on the interim assessment. We will then apply a second filter to sort students’ scores on the Iowa Assessments from “smallest to largest.” Then we will have Excel count the number of students who did not pass the Iowa Assessments and put that number in Cell C. Then, we will have Excel count the number of students who passed the Iowa Assessments and put that number in cell D.

 

Formulas are already in the Excel spreadsheet that will calculate the negative and positive predictive power and the overall classification index of the interim assessment relative to the outcomes measure (Iowa Assessments). Definitions are given for each in the Excel sheet.


Step 1: Add filters to your columns

  • Highlight/Select each cell that will be sorted/filtered (aka—the heading cells)
  • Under the “Home” tab within the “Editing” section click on the “Sort & Filter” and then select the “Filter” option:

Visually showing the process of the “Sort & Filter” and the “Filter” options selected

  • Each of your column heading cells should now have a sort/filter button attached:

Picture of column heading cells with sort/filter buttons


Step 2: Determining the numbers that go into cells A and B

Interim Assessment: Skills Iowa Outcome Measure: Iowa Assessments
Fail: Outcomes Assessment Pass: Outcomes Assessment
At Risk (Fail) Cell A = Cell B =
Not At Risk (Pass) Cell C = Cell D =

 

  • First we will need to filter the “Interim Assessment: Skills Iowa” column which contains the students’ Skills Iowa scores.
  • Click the filter button within that column’s heading cell; select “Number Filters” and then the “Less Than or Equal To…” option:

Visually showing the process of selecting the filter button, “Number Filters” option, and then the “Less Than or Equal To…” option

  • A “Custom AutoFilter” box will appear.  Choose the “is less than or equal to” option, type “74,” and then click “Ok.”

Visually showing the “Custom AutoFilter” box, with the “is less than or equal to” option, and typed “74,”

NOTE: All scores that were 75% or higher are now hidden from view within our Excel document.

  • We will now sort the “Outcome Measure: Iowa Assessments” column which contains the students’ National Standard Scores.
  • Click the sort/filter button within that column’s heading cell; select the “Sort Smallest to Largest” option:

Visually showing the process of clicking the sort/filter button within a column’s heading cell; then selecting the “Sort Smallest to Largest” option

NOTE: Students who received failing scores* within the Iowa Assessments will appear within the first rows of your Excel document.

 

*If you need the cut scores for Iowa Assessments, review the following document:  Standard Score Ranges for Achievement Levels

 

  • Let Excel count the number of students who failed both the Skills Iowa and the Iowa Assessments.  Simply highlight/select the scores that indicate a failing score on the Iowa Assessments.  At the bottom of your Excel document there will be a “Count” which provides the total number of highlighted cells.  Make sure that you only highlight within one column when you want Excel to count the number of scores.

In this example there are 31 students who failed both the Skills Iowa and the Iowa Assessments. This number (31) goes in cell location A.

Visually showing the process of selecting the 31 students who failed both the Skills Iowa and the Iowa Assessments.

  • Let Excel count the number of students who failed the Skills Iowa but passed the Iowa Assessments.  Simply highlight/select the scores that indicate a passing score on the Iowa Assessments.  At the bottom of your Excel document there will be a “Count” which provides the total number of highlighted cells.

In this example there are 9 students who failed the Skills Iowa but passed the Iowa Assessments.  This number (9) goes in cell location B.

Interim Assessment: Skills Iowa Outcome Measure: Iowa Assessments
Fail: Outcomes Assessment Pass: Outcomes Assessment
At Risk (Fail) Cell A = 31 Cell B = 9
Not At Risk (Pass) Cell C = Cell D =

 


 

Step 3: Determining the numbers that go into cells C and D

  • First we will need to undo the filter that we placed on the “Interim Assessment: Skills Iowa” column which contains the students’ Skills Iowa scores.
  • Click on the sort/filter button within that column’s heading cell; check the box in front of the “(Select All)” option and then click “Ok”

Visually showing the process of clicking the sort/filter button within a column’s heading cell then checking the box in front of the “(Select All)” option

 

NOTE: All scores that were 75% or higher have now reappeared within our Excel document.

  • Next we will filter the “Interim Assessment: Skills Iowa” column which contains the students’ Skills Iowa scores.
  • Click the filter button within that column’s heading cell; select “Number Filters” and then the “Greater Than or Equal To…” option:

Visually showing the process of clicking the filter button within a column’s heading cell; selecting “Number Filters” and then the “Greater Than or Equal To…” option

 

  • A “Custom AutoFilter” box will appear.  Choose the “is greater than or equal to” option, type “75,” and then click “Ok.”

Visually showing the “Custom AutoFilter” box with the “is greater than or equal to” option selected and "75" typed in

 

NOTE: All scores that were lower than 75% are now hidden from view within our Excel document.

  • We will now sort the “Outcome Measure: Iowa Assessments” column which contains the students’ National Standard Scores.
  • Click the sort/filter button within that column’s heading cell; select the “Sort Smallest to Largest” option:

 

Visually showing the sort/filter button within a column’s heading cell; with the “Sort Smallest to Largest” option selected

 

NOTE: Students who received failing scores within the Iowa Assessments will appear within the first rows of your Excel document.

 

  • Let Excel count the number of students who passed the Skills Iowa but failed the Iowa Assessments.  Simply highlight/select the scores that indicate a failing score on the Iowa Assessments.  At the bottom of your Excel document there will be a “Count” which provides the total number of highlighted cells.

In this example there are 3 students who passed the Skills Iowa but failed the Iowa Assessments. This number (3) goes in cell location C.

  • Let Excel count the number of students who passed both the Skills Iowa and the Iowa Assessments.  Simply highlight/select the scores that indicate a passing score on the Iowa Assessments.  At the bottom of your Excel document there will be a “Count” which provides the total number of highlighted cells.

In this example there are 32 students who passed both the Skills Iowa and the Iowa Assessments.  This number (32) goes in cell location D.

Interim Assessment: Skills Iowa Outcome Measure: Iowa Assessments
Fail: Outcomes Assessment Pass: Outcomes Assessment
At Risk (Fail) Cell A = 31 Cell B = 9
Not At Risk (Pass) Cell C = 3 Cell D = 32

Overall classification rates of around 80.0% are desired.


Conclusion

There is strong evidence that students who start at a lower level in reading seldom catch up during later school years (Francis, Shaywitz, Stuebing, Shaywitz and Fletcher, 1996; Juel, 1988; Torgensen and Burgess, 1998) and that catching these students up in later grades takes much more time and resources. This strategy, however, is much less successful (Foorman, Breier, and Fletcher, 2003; Torgesen, 2000). It is possible that by simply adjusting cut points for risk identification on interim assessments, or adding additional questions, or increasing the rigor on existing questions as called for in the Iowa Standards on teacher developed interim assessments, a different level of classification will be observed that better meets the needs of students.

 

Ensuring that fewer students are misidentified as not at risk and ensuring that those that are identified as at risk receive the MTSS guidelines of allocating resources for early identification and prevention will help students get what they need to become successful learners.