June 2004 — Exclusive Series: SBR
Print this articleClick here to receive your FREE subscription to T.H.E. Journal
What Does SBR Mean for Education Technology?
Outcomes and Test Instruments. Technology in today's schools takes many varied forms and addresses virtually every educational function and need - from delivery of core curriculum to changing the system of instruction. As a result, any technology evaluation must clearly articulate outcome goals and employ appropriately sophisticated outcome measures and data-gathering techniques to capture the technology's intended impact(s).
However, the year-end standardized achievement tests that serve at the core of NCLB's accountability requirements, and are driving much of education, often provide too narrow and blunt a measurement to adequately capture technology's impact. While SBR d'es not explicitly require use of this outcome measure, it is often the only one of interest to educational decision-makers whose schools' fates may rise and fall on student performance on such tests. There is concern that this focus will undercut the potential promise of the technology as a transformative productivity and learning tool. It will often fail to adequately capture important educational goals, including:
1. 21st century learning goals such as technology literacy, self-directed learning and problem-solving;
2. Indirect benefits in terms of changing the school process, improving efficiency and expanding opportunity; and
3. Core academic knowledge and skills that may not be measured in the finite scope of some standardized tests.
Do the current measures confine us to too narrow a form of learning and assessment? Is increased student achievement only demonstrated by high-stakes performance? Rather than focusing SBR only on improved standardized test scores, education should enhance the depth and breadth of its educational measures to accurately capture many of technology's profound impacts.
Research Models. The SBR definition, especially in the NCLB provisions for "proven effective by SBR," places a premium on evaluation studies capable of demonstrating causation. The NCLB definition, as well as the WWC research design standards, states that causation and effectiveness are best demonstrated through, at a minimum, quasi-experimental designs that employ control groups, preferably through experimental designs that also incorporate randomization. This raises a number of challenges.
First, as important as it is to know whether studies prove a product effective, educators need research to help them better understand how to replicate its successful use. Experimental design, by definition, tries to isolate all possible variables (e.g., class size, teacher preparedness, etc.) to zero in on the intervention itself. The problem, however, is that if a school wants to use the product, it needs to know the circumstance under which the product worked best. D'es class size matter in implementing a particular program? In the case of technology, d'es Internet access speed affect successful implementation? Experimental design, because it "cancels out" all other variables, d'es not offer this information to educators.
Second, many practical challenges arise when implementing studies that employ control groups and/or randomization of subjects. For example, what if students are grouped according to certain academic characteristics, and therefore random assignment is not possible? Or, what if the technology delivers the core curriculum, making it impossible to employ control groups where students would be denied the experimental intervention? While there are solutions to these dilemmas, the result is often a more challenging and less ideal research design.
