April 2004 — Exclusive Series: SBR
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Checklist to Use in Evaluating Whether an Intervention Is Backed by Rigorous Evidence
A. Circumstances in which a comparison-group study can constitute "possible" evidence:
- The study's intervention and comparison groups should be very closely matched in academic achievement levels, demographics, and other characteristics prior to the intervention.
- The comparison group should not be comprised of individuals who had the option to participate in the intervention but declined.
- The study should preferably choose the intervention/comparison groups and outcome measures "prospectively" — i.e., before the intervention is administered.
- The study should meet the checklist items listed above for a well-designed randomized controlled trial (other than the item concerning the random assignment process). That is, the study should use valid outcome measures, report tests for statistical significance, and so on.
B. Studies that do not meet the threshold for "possible" evidence of effectiveness include: (i) pre-post studies; (ii) comparison-group studies in which the intervention and comparison groups are not well-matched; and (iii) "meta-analyses" that combine the results of individual studies which do not themselves meet the threshold for "possible" evidence.
Step 3. If the intervention is backed by neither "strong" nor "possible" evidence, one may conclude that it is not supported by meaningful evidence of effectiveness.
Cite this Site
"Checklist to Use in Evaluating Whether an Intervention Is Backed by Rigorous Evidence," T.H.E. Journal, 4/1/2004, http://www.thejournal.com/articles/16717
copy text (above) for proper citation
