September 1996 — Editorial

Print this article | Email this article

Click here to receive your FREE subscription to T.H.E. Journal

Courseware, Assessment and Evaluation

Room for Improvement However, what passes as evaluation is often limited in both scope and scale. Though monitoring and assessment techniques are often embedded within software programs, these are frequently trivial, do not involve the end user, and therefore have not been properly tested. In a paper presented at the Educational Multimedia and Hypermedia conference in Boston, June 1996 by A. Bartalome, Universitat de Barcelona, and L. Sandals, University of Calgary, titled "Evaluating Educational Multimedia Programs in North America," they reviewed a small sample (26 sites) of end users' involvement in the development of educational multimedia projects. The 26 programs represent the work of more than 160 people during an average time of two years. In several cases, work is in progress on new versions. More than half of the programs were related to Science and Technology. (Though questionnaires were sent to more than 100 projects, the authors said its length and in-depth questions may have discouraged a greater response.) Even with a caveat, the paper's following conclusions are interesting: Educational multimedia programs were evaluated during their production (65%) and at the end of the production (68%). Most programs were continuously evaluated during production, but participation of the end user was not always encouraged. In the programs themselves, 92% include some type of activity or question (exercises, questions or problems to solve). Programs also included (a) a help system, (b) user control over the program, (c) a variety of levels for different users, (d) an assessment or evaluation system, and (e) a feedback summary for users. Note that 12% of the programs had less than three of the above "quality indicators" and the latter three (c, d, e) are usually given the least attention.
Design and creation of good software that includes worthwhile assessment tools that do more than report and critique responses requires significant allocation of resources and involves implementation on a significant scale. Research in application and design must remain a key issue. Technology, however sophisticated, plays only a small part in the complex learning process, but d'es provide the tool to assist us in our efforts.