January 2004 — Exclusive Series: SBR
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Determining 'What Works' - An Interview With Dr. Grover 'Russ' Whitehurst
T.H.E.: How d'es the Clearinghouse's work relate to the definition of scientifically based research as defined in the No Child Left Behind act?
WHITEHURST: Certainly a [research] study that meets the standards of the What Works Clearinghouse would be consistent with [NCLB's] definition of scientifically based research. However, the question addressed by the What Works Clearinghouse is a particular question, and that's 'what works'? There are many scientific questions that are not 'what works' questions. There are, for example, questions of what correlates with what: are poor people less likely to get appropriate funding for their schools than people from affluent neighborhoods? There are questions having to do with the validity of a test or assessment instrument. There are questions having to do with the way the brain works as it processes what an individual reads. All of those are questions that are addressed by science and fit the [NCLB's] broad definition of scientifically based research.
But the What Works Clearinghouse is only addressing one of those questions, and that's the question of effectiveness. And the effectiveness question requires a particular subset of standards that are much narrower than the general standards for scientifically based research. So a standard of the What Works Clearinghouse is that it privileges, or gives preference to, randomized trials because randomized trials are the gold standard for determining effectiveness.
T.H.E.: The Clearinghouse looks at research in a given topic area on the effectiveness of educational interventions, which are defined by the WWC as a product, practice, policy or program — what your site refers to as the 'four Ps.' Would you consider just one research study on a particular P or d'es there have to be a body of studies?
WHITEHURST: We would consider one study. In most cases, there will not be a body of research for a single P. If you look at the U.S. Food and Drug Administration as a model, what's required to get FDA approval to market a product are two randomized trials. So, we will privilege randomized trials. We will provide a registry of those trials as related to particular Ps. If there is only one such trial, there will be appropriate warnings and caveats that this is just one study and every study has limitations. Those limitations will be described in the evidence reports (see Pages 36-37 for more on evidence reports). But we think it is very important to provide the data that exists with respect to particular Ps. The body of evidence that you refer to is one that exists with respect to a body of related Ps.
T.H.E.: A body of evidence would be especially true for practices and policies, but not so much for products?
WHITEHURST: Well, it will be [true] in products as well. That is, there will be a variety of products that focus on middle school math. One can imagine an evidence report that focuses on what might have been common to the products that seemed to be more successful versus less successful. That's what the body of evidence would generate. At the same time, a local school administrator would want to know whether there is any evidence with regard to the effectiveness of say, 'Everyday Math,' and we intend to provide that evidence, which is likely to[come from only] one or two studies.
