September 2005 — Industry Perspective
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Test Bias or Real Differences?
A test publisher’s perspective on closing the achievement gap.
Even with the years of effort dedicated
to equalizing education, the
achievement gap in the US persists.
The question now facing test developers is
whether the gaps are the result of true
differences in achievement or of bias in the
tests. Test publishers have developed
methods of detecting and eliminating bias
in their assessment products; however,
despite all the attention and resources
directed at closing the achievement gap, a
disparity in performance still remains. But
an “item design” would enable assessments
not only to measure a student’s academic
achievement, but also to provide insight
and detailed information about what
learning objectives a student has not yet
reached. By both serving as a thermometer
and offering educational prescriptions,
standardized assessments can be a tool for
narrowing the achievement gap.
With professional and ethical responsibilities to remove barriers of bias in their assessments, test publishers extensively rely on industry standards to produce highquality assessment instruments. For a published test to be fair and unbiased, it must measure a student’s achievement without being affected by extraneous factors such as student gender, culture, ethnicity, geography, or soci'economic status. Deeply underpinning this discussion is that the clear purpose of educational assessments is to measure the differences in student achievement. There is no legitimate reason to build and administer a test that confirms sameness. Only by understanding the differences between students can we customize instruction for each student.
Given best practices in the industry for eliminating bias, what can test publishers do to contribute and facilitate a deeper understanding of the root causes of these differences? Addressing this requires a shift in the traditional function and design of standardized assessment systems and individual items. What if items were constructed differently so that they would reveal where learning breaks down for individual students?
Multiple-choice items have a long history of helping educators and policymakers understand what students know and can do. Basically, items are created to sort students into two categories: those who know the content and those who do not. They are not written to provide systematic insights into where students are in their thinking; valuable information for teachers whose job it is to help move students who do not answer the item correctly to an understanding of the standard.
Harcourt (harcourtassessment.com) has developed a new item type that helps teachers understand not only which students know the content, but also where the students who do not know the content are in their understanding. Given a typical multiple-choice item, we have the opportunity to sort students into four levels of achievement. By building assessments that provide increasingly precise information about why students choose the incorrect answer, test publishers make a direct contribution to the improvement of classroom instruction. Teachers can know where learning has broken down and where instruction should be targeted.
