January 2004 — Exclusive Series: SBR
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Determining 'What Works' - An Interview With Dr. Grover 'Russ' Whitehurst
The second point I would raise is that the idea that it's really expensive to develop evidence of effectiveness, I think, is wrong. I've had people come through my office and show me the evidence that they've collected on the effectiveness of their products, and in many cases these have been very small firms that have done an evaluation for $60,000. Now, it's true that if the total income for the vendor is $200,000 a year, then spending $60,000 a study is not practical. But, I think collecting evidence on effectiveness is not a huge business expense, except at the very lowest end. And usually there you're talking about brand new products, innovations where there is nothing else that's comparable, so you're not up against competitive pressure that somebody else has got the same widget that you've got and they've got evidence that you don't.
T.H.E.: In NCLB's definition of SBR, it says that research must be published in a peer-reviewed journal. D'es the WWC have the same requirement?
WHITEHURST: No.
T.H.E.: If the organization that conducted the research was hired by the publisher, d'es that diminish the research's validity in the eyes of the WWC?
WHITEHURST: Not necessarily, though independence is good.
T.H.E.: If a vendor claims that their product meets SBR standards, how d'es a school district know if that's true?
WHITEHURST: They go to the What Works Clearinghouse, click on that product and find out what the evidence report, what the study report, says.
T.H.E.: And if there is none there?
WHITEHURST: Then the vendor is lying.
T.H.E.: I hate to push you on this, but this is working as an approved list. It's functioning that way even if it isn't officially one.
WHITEHURST: No, it's not an approved list. That is, there is nobody at the U.S. Department of Education who is requiring that districts makes choices off products that appear with evidence of effectiveness in the What Works Clearinghouse.
T.H.E.: But if funding is tied to them, then they are.
WHITEHURST: Well, we don't even have any products yet, [so] how could they be making those decisions.
T.H.E.: But eventually you're going to be having those products?
WHITEHURST: But there's no regulation or guidance that's come out of the U.S. Department of Education's offices that provide funding to states and local education agencies that has said, 'If you get money here, you have to pick a P that has evidence of effectiveness that's listed in the What Works Clearinghouse.'
T.H.E.: Do you think that day will come?
WHITEHURST: I think that — to the extent that the What Works Clearinghouse provides a timely, user-friendly and generally unimpeachable evaluation of the evidence — increasingly people will use it, including the U.
