February 2001 — Features

Print this article

Click here to receive your FREE subscription to T.H.E. Journal

Assessing the Impact of Instructional Technology on Student Achievement

by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah Watson-Acosta

In January 2000, the survey was administered to 165 students in nine cooperating schools. One-hundred and thirty-seven responses were from students who had not yet been exposed to the intervention, and could therefore be used as pretests. Internal consistency and reliability for all scales (class motivation, school motivation, metacognition, inquiry learning, and application of skills) ranged from alpha = .70 to alpha = .87. In May 2000, at the end of the spring term, the survey was re-administered as a posttest to the same group of students. As of August 2000, 131 completed surveys were returned by all nine schools. About 75% of the students who responded were from high schools, and 25% were from middle schools. Gender was about equally distributed.

Seventy-six valid data sets were matched in order to conduct a true repeated measures methodology (pretest vs. posttest). Only the "application of skills" scale increased during the spring term (2-tailed significance = .0165).

For the path analysis, the posttest survey results were correlated with teacher assessments. Participating teachers assigned a "product" score of "0" (no evidence), "1" (approaches standards), "2" (meets standards), and "3" (exceeds standards) to their students' final products. Products were re-scored by a jury of experts to increase reliability, resulting in 91 reported "product" scores. One-hundred and seven teachers assigned a "process" score of "1" (low) to "4" (high) to each of their participating students for the quality and depth of revisions of their final products, which they construed as a measure of student learning processes. These data constituted two independent measures of student achievement, which served to complete the model.

Four separate simplified path analysis models were tested. The first pair addressed process and product outcomes for class motivation, and the second pair addressed school motivation. The statistically significant (p < .05) results were as follows:

  • Motivation was related to metacognition. The relationship between class motivation and metacognition was slightly stronger (R = .307, p < the relationship between school motivation and metacognition (R = .282, p < .0001).
  • The relationship between metacognition and inquiry learning (Beta = .546, p < .0001) was stronger than the relationship between metacognition and application of skills (Beta = .282, p < .0001).
  • The relationship between inquiry learning and the student learning process outcome (Beta = .384, p = .001) was stronger than the relationship between application of skills and the student learning process outcome (Beta = -.055, not significant).
  • The relationship between application of skills and the student product outcome (Beta = .371, p = .004) was stronger than the relationship between inquiry learning and the student product outcome (Beta = .063, not significant).

Clearly, correlation d'es not imply causality. However, when each of these elements was considered as an independent variable, there was a corresponding change in associated dependent variables. For example, there was a significant correlation between motivation and metacognition, indicating that students' enthusiasm for learning with technology may stimulate students' metacognitive (strategic) thinking processes.