January 2004 — Exclusive Series: SBR
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Determining 'What Works' - An Interview With Dr. Grover 'Russ' Whitehurst
T.H.E.: OK, let's say that the body of evidence finds that one of the common successful elements in a group of elementary math products was the use of manipulatives. I am in a district and use manipulatives in my elementary math instruction. D'es that mean I'm doing 'what works' or that I'm using a product that has a manipulatives component? What d'es that mean from the viewpoint of the WWC?
WHITEHURST: That's precisely why I said we are going to have a focus on individual studies that examine the effectiveness of particular products. Because the issue you raise is a very important issue that will arise if all we have are these broad evidence reports. [The fact] that there were particular products that involve manipulatives that work d'es not mean that manipulatives work as the local district is using them. We'd like to provide, to the extent possible, more detailed information at the level of the particular practice that a district might be using so the district can determine if there's evidence on whether that practice works.
T.H.E.: In other words, the body of evidence says that the use of math manipulatives seems to be effective based on this group of studies, and a particular product study evaluation would say, for example ...
WHITEHURST: ... that 'Everyday Math' works. And one of the characteristics of 'Everyday Math,' as described by the developers, is that it involves manipulatives. At that level, you are not saying that manipulatives work. You are saying there's evidence that when 'Everyday Math' was employed in real schools there was a sign of positive impact on student scores.
T.H.E.: If I am using a product that d'es not show up in your evidence report, but it uses manipulatives, how can I know for sure that it works?
WHITEHURST: You can't.
T.H.E.: You can't?
WHITEHURST: [Not] unless you collect the evidence yourself.
T.H.E.: How do you do that?
WHITEHURST: We think it's not only possible but highly desirable for school districts to collect performance data on any P they choose, whether evidence of its effectiveness is validated by the What Works Clearinghouse or not. Because an educational intervention is not the same thing as a pharmaceutical, something may work in one location, and circumstances may be so different in another school system that it might not work.
T.H.E.: I am assuming that the districts would be held to the same rigorous standards for SBR that everyone else is held to?
WHITEHURST: No, I am suggesting that there are two prongs to evidence-based practice*. One is relying on well-designed, scientific studies of effectiveness of the sort that will be vetted on the What Works Clearinghouse. The other is locally collected performance data that indicates whether changes are occurring in the desired direction when a particular program or practice is implemented. That's something that schools should do; it's not that hard to do. You've got a system in place for collecting performance data and you should do that.
