Computing Technology for Math Excellence Logo

 

 

 

 

 

 

 

 

Black line

Publications

Cite any reference to the article below as:

Deubel, P. (2001, Spring). The good and bad news of software use for mathematics proficiency test preparation. Ohio Journal of School Mathematics, 43, 55-60.

Black line

 

The Good and Bad News of Software Use for Mathematics Proficiency Test Preparation

 

Patricia Deubel, Ph.D.

Problem

In March 2000, 126,252 public school eighth graders took the Ohio Ninth Grade Proficiency Test (ONGPT). Approximately 46% passed all tests taken, a decrease of 3% from last year’s performance. Pass rates for this year’s eighth graders were lower on all five tests, with the biggest drops in writing (7%) and reading (5%). Mathematics results showed an overall decline of 1% from last year's passing rate (ODE, 2000b). Typically, mathematics has been the most difficult part of the ONGPT for students to pass. The problem can potentially get worse. Ohio's Senate Bill 55, the academic accountability package passed by the legislature in 1997, requires that beginning in 2003 the state's proficiency tests shall measure student knowledge of core academic areas through a tenth-grade level. The first class to be impacted by this new law is the Class of 2005 (ODE, 1998).

In the Ohio Department of Education (ODE, 2000a) report, The State of Education in Ohio, State Superintendent Zelman said that proficiency test results suggest that Ohio needs better approaches in mathematics instruction. In terms of the use of software to prepare students for the proficiency tests, however, the ODE's assessment center does not recommend products to school districts for preparation for the state's proficiency tests. School districts are told: Let the buyer beware (Gillespie, 1999).

Standards adopted by The National Council of Teachers of Mathematics (NCTM, 1989) and goals of the Ohio Schools Technology Implementation Task Force (1999) include that teachers should use electronic products to improve student learning. Students should have access to technology that will help them meet high standards of achievement. Has software use, however, made any difference on students passing the mathematics test of the ONGPT?

To address that question, Grade 8 mathematics, special education, and proficiency intervention teachers (N =113, which was an 88% response rate) in 35 middle schools across 13 mid-sized districts in the Urban Schools Initiative were surveyed prior to the March 2000 tests. The survey addressed teacher beliefs on individual and organizational factors related to the use of software during class time, what software was used, how it was used, and software's instructional and technical merit for proficiency test preparation. Mathematics achievement was measured for over 6000 Grade 8 students. The study received support from the ODE Urban Schools Initiative, which is now the Office of Comprehensive School Improvement. This article presents major findings of the study. The news is both good and bad.

Good and Bad News of Software Use

The good news is that software use during math class time appears to have increased among all 13 participating school districts from 1994-2000. Reported use from teachers showed a rise from 3% in 1994-95 to 52% in 1999-2000. The bad news is that software use during class time was not an integral part of the mathematics curriculum in most settings for those districts. Forty percent of participants had never used software in their instruction, which does not necessarily mean that teachers were resistant, incompetent, lacked expertise, or were technophobes, according to Cuban (O'Neil, 2000). Typical among the 52 reasons that teachers gave for not using software for instruction were statements such as:

Top test improvement strategies that all teachers used for proficiency test preparation included small group tutoring (81%), whole class instruction (75%), individual tutoring (71%), communicating high expectations (56%), followed by using computer software (50%). Even among the 52% of teachers who used software during 1999-2000, most only used it occasionally all year (43%), which meant at most a few times a month (50%).

The occasional use of software during class time actually had a significant negative impact on students passing the test. Software may not have been used in a focused manner to help individual students to master specific skills in which they were weak. More students who had not used software during class time passed the test than those who did use software during class time, a significant difference (Table 1).

Software use made a significant difference on passing the test for students who did not use software during class time, but had used software in proficiency intervention classes that met in addition to regular math classes. Unfortunately, most schools did not have the luxury of offering such classes. The implication is that the personalized, regular and focused effort on using software over time to combat individual weaknesses and to build skill fluency led to achievement gains (Table 2).

 

Table 1: Chi-Square Analysis of Test Results for Class Time Software Use

Actual

Expected

Passed

Did Not

Total

% Passed in Group

To Pass

Not Pass

 

Software

1129

1706

2835

40%

1264

1571

No Software

1991

2172

4163

48%

1856

2307

Total

3120

3878

6998

45%

 

Chi-Square:

43.714

Significant at .01 level

df =1

Probability:

3.8E-11

Note: Data for one school (N=294 grade 8) could not be included because results were reported for grades 8 and 9 combined. The 45% who passed the test included 16% who used software and 29% who did not use software.

 

Table 2: Chi-Square Analysis of Test Results for Class Time Software Use and Extra Proficiency Intervention Using Software

Actual

Expected

Software Use

Passed

Did Not

Total

% Passed in Group

To Pass

Not Pass

 

Class Use, No Extra

682

962

1644

41%

761

883

Class Use, W. Extraa

27

84

111

24%

51

60

No Class Use, No Extra

1150

1428

2578

45%

1193

1385

No Class Use, W. Extra

753

559

1312

57%

607

705

Total

2612

3033

5645

46%

 

Chi-Square:

104.808

Significant at .01 level

df =3

Probability:

1.4E-22

Note: This table includes data from 27 of 35 participating schools, which provided test results with a breakdown for students who had extra proficiency intervention using software.
a This category includes results from 3 teachers in 2 schools. In school 1, 9 of 18 passed (50%). In school 2, 18 of 93 passed (19%). Caution is needed in generalizing the category result because the sample size is too small in comparison to other groups.

 

 

So why haven't teachers used software more often in their instruction? Results indicated that administrative support, teacher instructional style, teachers' perceived priority of learning about computers and software, computer availability and access, availability of technical assistance when needed, and software quality were significant factors in teachers' decisions to use software in their instruction.

Surprisingly, source of computer learning, whether by staff development or learning on one's own, and lack of time to learn about computers and software were not factors that distinguished software users from non-users. Only 40% of teachers felt staff development met their needs to learn to use technology in the classroom. Likewise, only 19% of each group believed they had adequate time to learn about computers and new software. Anxiety was not an issue of concern. Only 12% of all teachers expressed anxiety when using computers and software.

Other good news is that teachers rated overall software quality as good for preparing students for the ONGPT math test. Ten of the top 12 software packages they used have correlation to NCTM, national and state standards, or to the mathematics learning objectives of the ONGPT (Table 3). The most frequently used program, Math Blaster (1997), was not among those, however. In fact, Soloway and Norris (1998) criticized Math Blaster for its gaming format. Students get to play a shoot-'em-up game having nothing to do with what they just learned as a reward for success. They proposed alternative software called Tenth Planet Explores Math (1996-2001). The fraction module, for example, provides multiple linked representations, and allows students to manipulate objects and to keep computer journals of their investigations. Problems are framed in real-world applications. The Tenth Planet fractions module makes much better use of the computational media than does Math Blaster, but the cost is four times as much for this thoughtful learning curriculum. Most software was used for remediation and drill and practice, but teachers clearly pointed out the need for drill and practice software for students who were failing the ONGPT math test because they lacked the skills tested on the exam.

 

Table 3: Top 12 Software Titles Used During Class Time

All Districts

(13, N=60)

>50% Districts (7, N=39)

<=50% Districts (6, N=21)

Title

Used

Best

Used

Best

Used

Best

 

Math Blaster (1997)

43%

3%

28%

0%

71%

10%

Passing the Ohio Ninth Grade Proficiency Test (1997)

37%

8%

33%

10%

43%

5%

Optimum (1997)

25%

5%

23%

5%

29%

5%

Math Skills and Concepts SuccessMaker (1999)

18%

5%

10%

5%

33%

5%

Hot Dog Stand (1996)

13%

0%

15%

0%

10%

0%

Math for the Real World (1996)

13%

2%

8%

3%

24%

0%

GoFigure (1997)

12%

5%

10%

8%

14%

0%

Math Munchers Deluxe (1997)

12%

0%

13%

0%

10%

0%

OPT for Success (1998)

12%

3%

13%

5%

10%

0%

Desktop Tutor (1996)

10%

2%

10%

0%

10%

5%

Mystery Math Island (1995)

10%

0%

13%

0%

5%

0%

SkillsBank (1998)

10%

0%

8%

0%

14%

0%

Note: All products, except Math Blaster and Hot Dog Stand, have been correlated to either national or state standards. The columns headed as Best indicate the percentages of teachers who chose the product as best for addressing the 16 learning objectives on the math test of the ONGPT. Districts were divided into two groups: those in which more than 50% of surveyed teachers used software and those in which 50% or fewer used software.

 

 

The bad news is that schools may not be getting the total return on investment of technology dollars. As one teacher in this study stated, software use was a pleasant break for kids. Ten of 36 software titles on the survey were titles that districts said were available, but were not used during math class time, although they may have been used during other times (e.g., tutoring sessions during or after school or during intervention classes). In fairness to teachers, some abandoned using available software because of its weaknesses. Some software dated back to 1991, or could not run on available computers. For some, building wiring and software licensing issues prevented access to software they wanted to use.

Valuable Software

Valuable educational software has stated learning objectives that are adhered to, is motivating to students, has well designed navigation icons, and contains multimedia features. It is also available above Version 1.0, which often has not been debugged and should be avoided (Abramson, 1998). Although overall software quality was rated good, findings revealed that software did not always have those valuable qualities, or the ability to foster higher order thinking skills.

Weaknesses in software's instructional and technical merit revealed that teachers want a management system that allows them to modify software to individualize instruction. The management system should keep track of student progress, identify areas of weakness and strength, suggest paths to improve, and have the ability to automatically adjust for student needs. The software should have more than one entry level, more than one level of difficulty, and provide some repetition to assist in retention. Software should allow students to save data, so that if they are not finished with a lesson, they do not have to begin again. Students should be able to change answers before software grades assessments.

Valuable software has an extensive database of problems, so that upon repeated use of software, students encounter a different set of problems. Problems should reference real-life applications and software should accommodate more than one solution method. Feedback should be tutorial in nature, not just indicate if responses were right or wrong. Help and audio features should be under user control. Teachers want software security against student errors or their intentional attempts to disrupt software operation. Software should meet NCTM, national and state standards, and learning objectives for the ONGPT. All of these criteria provide a set of guidelines to help districts in selecting software.

Conclusions

Clearly, much can be said about improving mathematics instruction using technology. Money continues to be an issue for many urban schools, particularly those that do not qualify for a lot of federal aid. One middle school principal stated their district needed computers, but fell in the middle between the haves and have-nots and must purchase their own technology. Although district and school budgets paid for the majority of software, teachers in 8 of 13 districts indicated some use of outside funding sources, including grants, parent associations, or Ohio School Net.

Schools cannot keep up with purchases of the latest software to run on older computers. Teachers cannot be expected to have contingency lesson plans for technology that fails when they need it. Therefore, they continue to use textbooks, overhead projectors, and chalk because they are reliable and flexible, as this study demonstrated. Let the buyer beware does not appear to be a sufficient answer when purchasing software, if the goal is to raise achievement of students with technology in Ohio's classrooms.

References

Abramson, G. W. (1998). How to evaluate educational software. Principal, 78(1), 60-61.

Desktop Tutor (Version 1.0) [Computer software]. (1996). Hollandays Publishing Corporation, 8459 North Main Street #118, Dayton, OH 45415. http://www.hollandays.net/
[Note URL and address update January 22, 2010.]

Gillespie, C. (1999, September 24). Firm offers a trial Ohio proficiency test on Net. Cleveland, OH: The Plain Dealer, p. 5B.

Go Figure [Video and Computer software]. (1997). WVIZ Channel 25 and Ohio University Public TV, 4300 Brookpark Road, Cleveland, OH 44134.  [June 4, 2014 Update: The original Go Figure project has been expanded and is available from WOUB Center for Public Media, 9 South College Street, Athens, Ohio 45701. http://woub.org/gofigure/ ].

Hot Dog Stand: The Works [Computer software]. (1996). Sunburst Technology Corporation, Customer Service: 1550 Executive Drive, Elgin, IL 60123.  Available from Learning Services, P.O. Box 10636, Eugene, OR 97440.  http://www.journeyed.com/products/Sunburst/Hot+Dog+Stand+Series      [Note URL and availability update August 5, 2016 no longer available.]

Math Blaster [Computer software]. (1997). Sunburst Technology Corporation, Customer Service: 1550 Executive Drive, Elgin, IL 60123. http://ka.sunburst.com/mb.php   [Note URL update June 4, 2013.]

Math Concepts and Skills SuccessMaker (Version 5.2) [Computer software]. (1999). Mesa, AZ: Pearson Education Technologies. http://www.pearsonschool.com/index.cfm?locator=PSZyM6   [Note URL for SuccessMaker and product availability updated July 28, 2011.]

Math for the Real World [Computer software]. (1996). Knowledge Adventure, 2377 Crenshaw Blvd., Suite 302, Torrance CA 90501. http://www.knowledgeadventure.com/school/othertitles/MFTRW.aspx  [Note URL and product availability last updated Jan. 5, 2015.]

Math Munchers Deluxe [Computer software]. (1997).  Riverdeep, Inc., Corporate Headquarters, 100 Pine Street, Suite 1900, San Francisco, CA 94111. http://web.riverdeep.net/portal/page?_pageid=813,1&_dad=portal&_schema=PORTAL  [Note company address, URL and product availability updated Jan. 22, 2010.]

Mystery Math Island [Computer software]. (1995). Lawrence Productions, Inc., 1800 South 35th Street, PO Box 458, Galesburg, MI 49053-0458.

National Council of Teachers of Mathematics Curriculum and Evaluation Standards. (1989). Reston, VA: National Council of Teachers of Mathematics. Available: http://www.nctm.org/standards/ [URL updated on Jan. 26, 2009.]

Ohio Department of Education. (1998). News release: Ninth-graders break overall, individual records on ninth-grade tests. http://www.ode.ohio.gov/www/ims/prof9_news_oct97.html
Updated January 26, 1998. Accessed July 22, 1999.  [Note at update Jan. 26, 2009, URL no longer available.]

Ohio Department of Education. (2000a, February). The State of Education in Ohio.

Ohio Department of Education. (2000b, June). News Release: Class of 2000 Maintains Steady Performance on Ninth-Grade Proficiency Test. http://www.ode.state.oh.us/comm/news/06-05-00.htm Updated June 2000. Accessed June 6, 2000. [Note at update Jan. 26, 2009, URL no longer available.]

Ohio Schools Technology Implementation Task Force report: Senator Robert R. Cupp, Chairman. (1999, March 3). pp. 1-35. http://www.ohioschoolnet.k12.oh.us/extlink/frameset.asp?exturl=/resources/document/documents/doclib_490.pdf&returl=/ Accessed July 8, 2001. [Note at update Jan. 26, 2009, URL no longer available.]

O'Neil, J. (2000). Fads and fireflies: The difficulties of sustaining change. Educational Leadership, 57(7), 6-9.

OPT for Success (Version 2.0) [Computer software]. (1998).  Kamico Instructional Media, 4413 Spicewood Springs Road Austin, TX 78759-8592 . http://www.kamico.com Email: info@kamico.com [Note:  This product is now known as Path to Mastery.  Address and URL for availability were updated on Jan. 26, 2009.]

Optimum (Version 1.0) [Computer software]. (1997). Photonics Graphics, 2244 Park Avenue, Cincinnati, OH 45206. http://www.optimumtest.com Email: photonics@photonicsgraphics.com

Passing the Ohio Ninth Grade Proficiency Test (Version 2.0) [Computer software]. (1997). Englefield & Arnold, Inc., 2121 Bethel Road, Suite D, Columbus, OH 43220. http://www.eapublishing.com

SkillsBank (Version 4) [Computer software]. (1998). Achievement Technologies, Inc. School Division: 10400 Little Patuxent Parkway, Suite 310, Columbia, MD 21044. http://www.achievementtech.com/    [Note URL and product availability update on Jan. 26, 2009.]

Soloway, E., & Norris, C. (1998). Using technology to address old problems in new ways. Association for Computing Machinery, Communications of the ACM, 41(8), 11-18.

Tenth Planet Explores Math [Computer software]. (1996-2001). Sunburst Technology Corporation, Customer Service: 1550 Executive Drive, Elgin, IL 60123. http://software.sunburst.com/index.htm#TENP  Last accessed June 4, 2014.  [Note URL update since original article.]

Contributor:

Patricia Deubel holds a Ph.D. in Computing Technology in Education from Nova Southeastern University in Florida, a Masters degree in Curriculum and Instruction from Ashland University in Ohio, and a B.S. in mathematics from Ohio University. She has over 25 years of experience in mathematics and computer education teaching, teacher training and staff development, and curriculum development. She has presented computer workshops at the state and local levels and currently teaches mathematics at The Ohio State University in Mansfield. Her recent articles have been published in the Journal of Instruction Delivery Systems and HyperNexus: Journal of Hypermedia and Multimedia Studies.

For further consultation regarding the outcomes of this study email: deubelp@neo.rr.com

back to top

*Follow-up note regarding Hot Dog Stand:  While in December 1999, Sunburst Technology Corporation (their new name) did not provide any correlations, correlations to national standards are now available at their web site,  according to B. Daly at Sunburst (email communication April 20, 2001).

Note (Jan. 26, 2009): Some URLs since publication of the article have changed, and are reflected in the updates.  Likewise, some products are no longer available, or developers have created newer versions.   See our section on math software for product availability.