February 2001 — Features
Print this article | Email this articleClick here to receive your FREE subscription to T.H.E. Journal
Assessing the Impact of Instructional Technology on Student Achievement
The WEB Project is a five-year Technology Innovation Challenge Grant that was completed in September 2000. The purpose of this project was to infuse standards-based instruction in multimedia, digital art, music composition, and online discourse into the general arts and humanities curricula of Vermont K-12 schools. Multimedia technology was incorporated within six academic content areas: art, music, technology, history/social studies, English/language arts, and interdisciplinary studies.
Students shared their works-in-progress with a virtual community consisting of other students, teachers, digital artists, traditional artists, musicians, composers, Web page designers and other experts. This was done via a virtual learning environment called The WEB Exchange, which resided on The WEB Project's server. Through threaded design conversations, students requested feedback on their works of art, music, and multimedia. They then filtered the feedback they received and used it to improve their final artistic products, which many of the participating students then posted on The WEB Exchange.
Concurrently, language arts students, aided by language arts teachers and mentors from the Vermont Center for the Book, discussed curriculum-related texts. Moderating their own discussions, students engaged in deep, rich dialogue that focused on standards-based activities such as responding to text, substantiating arguments with evidence found in the text, informed decision-making, etc. These interventions were stable over the last two to three years of the five-year grant.
Measuring the Project's Impact
One of the research questions posed by the RMC Research Corporation evaluation team in evaluating this project was, "What is the impact of The WEB Project on student achievement?" Our intent was to generalize our methodology to other instructional technology grants in which student achievement must be reported. Findings from an online survey of The WEB Project teachers and administrators, repeated in spring 1998, 1999, and 2000, indicated that a connection between student motivation, metacognition, and learning processes, as outlined in a conceptual model developed by Sternberg (1998), might be applicable.
According to Sternberg, motivation drives metacognition, which, in turn, stimulates the development of thinking and learning skills. Thinking and learning skill development further stimulates metacognition, resulting in the development of expertise. The evaluation team extended the Sternberg Developing Expertise model to define "expertise" as student achievement measured by teacher-created rubrics. Participating teachers developed, refined, and benchmarked rubrics for student-created products over the past three years. Teachers also selected a rubric for measuring student learning processes from Marzano et al.'s (1993) Dimensions of Learning model. The rubric addressed the depth and richness of revisions to student-created products and performances.
The Survey
Using mixed methods that consisted of the online survey, student pretest and posttest surveys, and scores on teacher-created/selected rubrics that assessed students' learning processes and final products, the evaluation team used structural equation modeling to correlate the various elements of the extended Sternberg model.
