Assessment Testing >> In Their Hands : February 2006 : THE Journal

February 2006 — Features

Click here to receive your FREE subscription to T.H.E. Journal

Assessment Testing >> In Their Hands

by Jacob Milner

Handheld devices empower teachers with assessment data they can put to immediate use.

At the Orange County Public School District in Orlando, FL, assessing reading skills among the youngest students used to be quite a process. Relying on rudimentary products such as paper and pencils, the strategy hinged on the bubble sheets teachers administered to students once a year. After teachers scored the exams, they sent them to the district office, where results were scanned, analyzed, and combined to form summary reports. These reports gave teachers information about which students needed extra help, and which subjects were proving to be troublesome. But because the reports took weeks to generate, it was difficult for teachers to use them to better serve the needs of their students.

Everything changed with the implementation of a three-year pilot program that kicked off the 2003-2004 school year. District officials, eager to improve their assessment techniques, turned to Wireless Generation (www.wirelessgeneration.com) to find a way to assess students so that teachers could actually do something with their data. Change came in the form of Palm handheld devices (www.palm.com). Teachers used them to record student performance on a series of questions designed to gauge reading skills. According to G. Lee Baldwin, the district’s senior director of Accountability, Research, and Assessment, improvements were seen virtually overnight. In the first six months, with teachers now able to respond to the dictates of the data, reading scores rose dramatically.

“This technology has eliminated the drudgery of assessment,” Baldwin reports. “We’re assessing our students more accurately, efficiently, and quickly.”

Orange County is not the first district to recognize the benefits of handheld-based formative assessment in grades K-3, when students are too young to take tests, and teachers assess them through observation. Across the country, other trailblazing school districts are getting in on the action too, downsizing assessment efforts into the palms of teachers’ hands. This new trend in assessment mixes software with portable hardware in a way that makes evaluating student skills unobtrusive and easy. Teachers give assessments to students one-on-one, and tap or write on the handhelds to record performance. With up-to-the-minute reporting applications, the technology also enables teachers to tabulate overall performance quickly, providing them with a virtually real-time picture on which students need help, where they need it, and how the teachers can help them best. While dozens of software companies sell tools that they describe as formative assessment (see “Other Options,” below), only a handful of firms sell formative assessment tools specifically for the handheld environment.

Two are New York-based Wireless Generation and Tango Software (www.tangosoftware.com), a division of Liberty Source in Austin, TX. It’s not a fair fight; Wireless Generation plays on the national scale, while Tango controls only a small portion of the Texas market. Still, as both vendors look to grow their market share in the months ahead, the formative assessment industry is emerging as one of the hottest and most exciting areas in educational technology.

Assessing Tech Skills

A new product helps judge the technological proficiency of elementary and middle school students.

Math and literacy aren’t the only skills teachers can evaluate with formative assessment tools; thanks to Portland-based Learning.com (www.learning.com), educators now can evaluate how deftly students can operate the latest and greatest technologies, too. The vendor recently unveiled TechLiteracy Assessment, an online authentic assessment of the technology aptitude of elementary and middle school students. The product, designed to monitor student progress toward both state and national education technology standards, will be available to all US districts starting in spring 2006. Pricing is $5 per student per year.

Gaston County School Districtin Gastonia, NC, was one of the districts that participated in a recent beta test of the product. Roxie Miller, the district’s assistant director of technology instruction, said recently that it’s a perfect tool for gauging assessment in her district, and should be equally useful for other districts down the road. “TechLiteracy Assessment had just what we were searching for,” she said. “An easy-to-use, online assessment aligned with state standards that assesses student knowledge of computer and technology skills in both multiple-choice and performance formats.”

“Handheld-based formative assessment is important in that you are able to set intermittent measures, or benchmarks, that let you adjust your instruction as you go along,” says Daniel Garcia, assistant superintendent for Curriculum and Instruction at Brownsville Independent School District in Brownsville, TX, which uses the solution from Tango. “The fact that the [technology] allows you to make immediate corrections/adjustments to instruction and add value by re-teaching as needed…is a plus.”

Forming Formative

Assessment in K-12 education is nothing new. Summative assessments—benchmark evaluations conducted at the end of an academic year—have been around for years in the form of big, scary exams. On the other hand, formative assessments, tests that monitor student performance throughout a portion of the curriculum (see “Formative vs. Summative,” p. 34), are relatively new to this decade . In the late 1990s, academic publishers introduced computer-based assessment tests to be administered multiple times during a school year. Since then, interesting trends in elementary assessment have developed: Grades 4-12 have emphasized summative but not so much formative assessments, while grades K-3 have focused more on formative.

Handheld formative assessment technology provides teachers with a virtually real-time picture on which students need help, where they need it, and how the teachers can help best.

Today, the bleeding edge of assessment is written for personal digital assistant (PDA) operating systems and is administered on handheld technology: various devices from Palm (www.palm.com), Hewlett-Packard (www.hp.com), and others. According to Naomi Hupert, senior research associate with the Education Development Center (main.edc.org) in Newton, MA, the approach monitors students on elements they need to master in order to move on to the next level of a particular subject. Hupert says formative assessment is a perfect way for teachers to catch and address gaps in a student’s knowledge or learning, early on. To this end, she adds that the technology provides teachers with the opportunity to teach more effectively.

Formative vs. Summative

The two types of assessment have different aims; one wants to develop student learning, the other wants to measure it. According to Paul Black and Dylan William’s seminal “Inside the Black Box: Raising Standards Through Classroom Assessment” (Phi Delta Kappan, October 1998), formative assessment is a powerful means of improving student learning. It is characterized by the effort of teachers to utilize technology to feed information back to students in ways that enable the students to learn better; it also describes the process of students engaging in a similar, self-reflective effort on their own. Formative assessment is particularly effective for students who have not done well in school, thus narrowing the gap between low and high achievers while raising overall achievement.

In contrast, summative assessment is the attempt to summarize student learning at some point in time; for example, at the end of a course. Most standardized tests are summative. They are not designed to provide the immediate, contextualized feedback useful for helping teacher and student during the learning process. High-quality summative information can, of course, shape how teachers organize their courses or what schools offer their students. The downside, however, is that if a summative assessment is administered too early in the learning process, it discourages teachers and students alike by assuming a knowledge gap.

“If there’s a content area where there is some underlying knowledge, students need to know it and have it available to them,” says Hupert, who also serves as a Reading First evaluator in New Mexico. (Reading First is the US Department of Education’s nationwide effort to enable all students to become successful early readers. See www.ed.gov/programs/readingfirst.) “The process of formatively assessing those students can become a valuable tool for teachers so they can be sure that the baseline of knowledge is being used appropriately by students.”

As Hupert suggests, handheld formative assessment systems don’t only assess student progress; the tests also can provide subtle evaluations of curriculum effectiveness. Here, teachers can use the assessment technology to gauge how well they’re teaching to student needs over time. If, for instance, an October assessment reveals a particular student is at risk of failing to grasp a particular subject, a teacher can reassess that student in December to see if changes in the curriculum have helped. If the student shows progress, the teacher can note what kind of approach to learning works best. If the student demonstrates little to no progress, the teacher knows it’s time to try something new.

Still, handheld-based formative assessment systems are not without a little controversy. For starters, some teachers say that the technology is disruptive, cutting into valuable classroom time that teachers could spend teaching. What’s more, because the latest formative assessments are generating precise and transparent data, many school districts see the tools as panaceas for assessment needs as a whole, and treat the formative tools as if they are more summative than they reallare.

“At the end of the day, the truth is that formative assess-ment is larger than merely assessing more frequently,” says Rick Stiggins, CEO of the Assessment Training Institute (www.assessmentinst.com) in Portland, OR.

The Trailblazer Hands down, Wireless Generation is the market leader in handheld-based formative assessment systems. The company was founded in 2000 with the goal of putting formative assessment into the hands of teachers and getting them away from more cumbersome materials such as desktops and paper. Today, the firm’s mCLASS (mobile classroom assessment) technology is used to assess 10 percent of the K-3 population in 48 states. Half of the Reading First students across the country are being assessed using its software. The company’s products constitute the official Reading First solution in 16 states.

Wireless Generation uses fixed, or preset, assessments to monitor student progress in two main subjects: reading and math. Reading is by far the larger of the two markets, with products specifically designed to assess pre-kindergarten literacy as well as K-3 student performance. This is accomplished via early-reading standards such as Dynamic Indicators of Basic Early Literacy Skills (DIBELS;dibels.uoregon.edu); Phonological Awareness Literacy Screening (PALS; pals.virginia.edu); Texas Primary Reading Inventory (TPRI; www.tpri.org); and El Inventario de Lectura en Espanol de Tejas or Tejas LEE (www.tejaslee.org), which establishes baselines for reading comprehension in Spanish. Math products, set to be released later this year, will cover early numeracy, operations, and all the basics of algebra.

When you get data to teachers in real-time, the entire learning process becomes more responsive in a way that motivates kids to get to the next goal.
Larry Berger, CEO, Wireless Generation

Whatever subject an assessment is evaluating, assessment systems from the predominant vendors work in pretty much the same way. First, a teacher administers a paper-based test and assesses student performance on the test using a handheld computer. When the assessment is complete, the teacher uses the device to look back at student performance as a whole. Next, the teacher puts the handheld device in its cradle and “syncs” all applicable data to the Internet. Finally, the teacher, coaches, and relevant administrators can log in through secure, password-protected access to view a variety of data reports and analyses and use instructional planning tools on a Web site.

“When you get data to teachers in real time, a bunch of things change,” says Larry Berger, CEO of Wireless Generation. “The entire learning process becomes more responsive in a way that motivates kids to get to the next goal.”

This is exactly what has happened at the Orange County PSD, where, at the start of the 2003 school year, district officials signed up for the DIBELS assessment software from Wireless Generation. The product, dubbed mCLASS: DIBELS software, offers teachers a specific K-6 assessment with progress-monitoring questions that meets requirements established by 45 Reading First states. Baldwin, the district’s director of assessment, says teachers in most of Orange County’s Reading First schools are required to utilize the technology four times a year, while those in other schools must use it three times. So far, he adds, reading scores have improved across the board.

Next »
1. 1
2. 2

THE Journal

February 2006 — Features