Computing Technology for Math Excellence Logo

 

 

 

 

 

 

 

 

Black line

Education Research:
Selected Research on Standards, Achievement, Learning, Assessment, and Technology Integration

 

Man with a stack of booksInternet World

This section for Education Research includes supporting pages of resources associated with State and National Standards.

Research Resources (Page 1 of 2) includes:

  • No Child Left Behind and Scientifically Based Research--a short essay
  • Efforts to Determine Product Effectiveness
  • Reading and Conducting Research, including Action Research
  • Finding Education Research on the Web

Arrow: You are hereResearch Resources (Page 2 of 2) includes summaries of selected research and resources related to:

Adobe Reader GifGet Adobe Acrobat Reader, free software for pdf files, which appear on this page.

Black line

Standards, Raising Achievement, Assessment, How People Learn

 

Numbers, Numbers, Test JPG

American Educational Research Association. (2000, July). Position statement concerning high-stakes testing in preK-12 education.

The American Educational Research Association (AERA) is the nation's largest professional organization devoted to the scientific study of education.  AERA's position statement on high-stakes testing is based on the 1999 Standards for Educational and Psychological Testing. The Standards represent a professional consensus concerning sound and appropriate test use in education and psychology. The statement includes that high-stakes testing should meet all of the following criteria:

American Federation of Teachers (2001). Making standards matter. Retrieved from http://www.aft.org/periodical/american-educator/winter-2001/making-standards-matter-2001

This report in four sections examined state efforts at setting standards, curriculum development, assessment, incentives, and intervention for students at risk of not meeting standards.  It does not address professional development.  The AFT made several recommendations.  For example, states should (p. 7):

Australian Council for Educational Research posted a number of relevant publications on brain-based research and learning: https://www.acer.edu.au/ 

Banilower, E. R., Smith, P. S., Weiss, I. R., Malzahn, K. A., Campbell, K. M., & Weis, A. M. (2013). Report of the 2012 National Survey of Science and Mathematics Education. Chapel Hill, NC: Horizon Research, Inc.  Retrieved from http://www.horizon-research.com/2012nssme/research-products/reports/technical-report/

The Report of the 2012 National Survey of Science and Mathematics Education "details the results of a survey of 7,752 science and mathematics teachers in schools across the United States. Areas addressed include: teacher backgrounds and beliefs, teachers as professionals, science and mathematics courses, instructional objectives and activities, instructional resources, and factors affecting instruction."  The entire report or selected chapters can be downloaded.  The following are among the multiple conclusions:

Barley, Z., Lauer, P. A., Arens, S. A., Apthorp, H. S., Englert, K. S., Snow, D., & Akiba, M. (2002). Helping at-risk students meet standards: A synthesis of evidence-based classroom practices. Aurora, CO: Mid-continent Research for Education and Learning. Retrieved from http://eric.ed.gov/?id=ED475904

This research synthesis provides evidence of five strategies to help low-achieving students meet standards:

Deans for Impact. (2015). The Science of Learning. Austin, TX: Deans for Impact.  Retrieved from http://deansforimpact.org/the_science_of_learning.html

In the Science of Learning, the Deans for Impact (2015) provide a valuable summary of cognitive science research on how learning takes place.  In it you'll find cognitive principles and practical implications for the classroom related to six key questions on how students understand new ideas, learn and retain new information, and solve problems; how learning transfers to new situations; what motivates students to learn; and common misconceptions about how students think and learn (About section).

Educational Leadership magazine, September 2001, is devoted to making standards work: http://www.ascd.org/publications/educational_leadership/sept01/vol59/num01/toc.aspx  ASCD has made some of those articles available to help you understand more about standards:

Gandal, M. & Vranek, J. (2001, September). Standards: Here today, here tomorrow. Educational Leadership, 59(1) 6-13.  Retrieved from http://www.ascd.org/publications/educational_leadership/sept01/vol59/num01/toc.aspx

Standards must be teachable to have an impact on what goes on in the classroom.  Teachable standards contain clarity and specificity.  In other words, they contain enough detail and precision so that teachers, students, and parents know what is to be learned.  Making decisions as to what all students should learn is difficult.  Including everything that a student could learn helps no one.  This "laundry list" tends to undermine the power of standards as common expectations.  Teachers feel overwhelmed by the enormity of what needs to be taught, and hence, in-depth coverage of concepts suffers.

According to Gandal and Vranek, those who design tests that measure state standards might consider three key principles:

Professional development linked to standards is key, but lacking for the majority of teachers in the United States. According to Education Week's (2001) Quality Counts survey, "fewer than half of teachers in the United States say that they have plenty of access to curriculum guides, textbooks, or other teaching materials or to specific training connected to state standards."

If standards truly define essential skills and knowledge that students should acquire, then students must be given a fighting chance to reach them, be it more time or extra targeted help on weaknesses through intervention and support programs.  Schools require a focused curriculum aligned to standards. Students should have adequate preparation time for tests and multiple opportunities to retake tests, if they do not succeed on a first try.

Gayler, K., Chudowsky, N., Kober, N., Hamilton, M., & Yeager, M. (2004, August). State high school exit exams: A maturing reform. Washington, D.C.: Center on Education Policy. Available under section: High School Exit Exams at http://www.cep-dc.org/

Gayler, Chudowsky, Kober, Hamilton, and Yeager presented findings and recommendations about exit exams in 2004, which is the third annual report on this topic.  In six chapters, the authors addressed how exit exams are affecting curriculum, instruction, and students; the characteristics of exit exams and how well they align with state standards; the kinds of exit exam supports and options that states are providing for students; the kinds of changes that states are making in exit exam systems; and how well exit exams are connected to other education policies such as No Child Left Behind.  State profiles are included.

Gayler, K., Chudowsky, N., Kober, N., & Hamilton, M. (2003, August). State high school exit exams: Put to the test. Washington, D.C.: Center on Education Policy. Available under section: High School Exit Exams at http://www.cep-dc.org/

Gayler, Chudowsky, Kober, and Hamilton presented findings and recommendations about exit exams based on a survey study of 24 states with current or planned exit exams.  This five chapter report, which is supported by review of literature on relevant studies of the prior year, addresses how exit exams are affecting curriculum, instruction, and students; the main features of state exit exams as they existed in 2003; the cost of implementing state exit exams; and the challenges that states face as they implement exit exams.  This report also includes state high school exit exam profiles (data and descriptions) for all 24 states that have or plan to have mandatory exit exams by 2008. Test types administered are either standards-based, minimum competency, or end-of-course.

Among findings:

Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009, April). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/PracticeGuide/2

The authors' "goal in this practice guide is to provide suggestions for assessing students’ mathematics abilities and implementing mathematics interventions within an RtI framework, in a way that reflects the best evidence on effective practices in mathematics interventions" (p. 4).  Eight recommendations are included, along with a list of 12 examples of math problems to illustrate concepts.

Ito, M., Horst, H. A., Bittanti, M.,  Boyd, D., Herr-Stephenson, B., Lange, P. G., Pascoe, C. J., & Robinson, L. (2008, November).  Living and learning with new media: Summary of findings from the digital youth project. The John D. and Catherine T. MacArthur Foundation Reports on Digital Media and Learning. Retrieved from http://digitalyouth.ischool.berkeley.edu/report

The authors presented findings from an ethnographic study, called "the most extensive U.S. study of youth media use" to date.  It was conducted over three years (2005-2008). More than 800 youth and their parents were interviewed by a team of 28 researchers and collaborators at the University of Southern California and the University of California, Berkeley.  The team spent more than 5000 hours observing teens involved with social networking sites, and examined diaries of youth who documented their everyday use of digital media. Results contribute to an understanding of how young people learn and develop social skills online and have implications for educators.

Major findings included:

Implications included:

Kauffman, D., Johnson, S. M., Kardos, S. M., Liu, E., & Peske, H. G. (2002, March). Lost at sea: New teachers' experiences with curriculum and assessment. Teachers College, Columbia University: Teachers College Record, 104(2), 273-300. Retrieved from http://www.tcrecord.org/Content.asp?ContentID=10822

Research that included semi-structured interviews and brief surveys was conducted in 1999-2000 with a diverse group of 50 first and second year Massachusetts teachers working in a wide range of  public schools to uncover their experiences with curriculum and assessment.  Kauffman and colleagues indicated the results of the study might not be generalizeable to the nation as a whole, but do have an implication for practice.

In terms of the interaction of state standards and curricula, research revealed:

The authors recommended actions in three arenas: state policy, curriculum research and development, and collaboration around curriculum at the school site. Beyond developing standards and assessments, states must support their implementation.  Research is necessary to evaluate the effectiveness of existing materials, the conditions under which those materials will be effective, and to develop additional materials for use in various academic subjects.  School-based collaboration around curriculum development would go a long way towards orienting new teachers to the curriculum and helping them learn what to teach and how to teach it.

Marzano, R. (2003). What works in schools: Translating research into action. Alexandria, VA: ASCD. ISBN: 0-87120-717-6.

Marzano synthesized 35 years of research on effective schooling and identified 11 school, teacher, and student level factors that have the greatest affect on student achievement:

School Level

  • Guaranteed and viable curriculum
  • Challenging goals and effective feedback
  • Parent and community involvement
  • Safe and orderly environment
  • Collegiality and professionalism

Teacher Level

  • Instructional strategies
  • Classroom management
  • Classroom curriculum design

Student Level

  • Home environment
  • Learned intelligence and background knowledge
  • Motivation

McIntosh, S. (2012, September). State high school exit exams: A policy in transition. Washington, D.C.: Center on Education Policy. Available under section: High School Exit Exams at http://www.cep-dc.org/

This 11th annual report on the state of high school exit exams is based on data collected from personnel in 45 state education departments.  Shelby McIntosh from the CEP reported three broad conclusions:

  1. High School exit exams remain a substantial force in education policy with 25 states administering such exams in 2011-2012.  Exit exams current affect nearly 7 of 10 students across the nation.
  2. Exit exams are becoming assessments of college and career readiness. Many states plan to use the Common Core State Standards, and often the common assessments, as a vehicle for this transition.
  3. Although the success of exit exam policies remains questionable, much can be learned from past implementations of new policies.  Namely, successful implementation of a new exit exam policy often depends on states' willingness to maintain the support of key state leaders and the public, implement the new policies over several years rather than all at once, and make a full financial commitment.  There are many challenges to a new policy implementation, such as "opposition from key education stakeholders, political disagreements or changes in state leadership, legal battles, low student passing rates, and high costs" (p. 1). "States have often responded to these challenges by offering alternate routes to graduation or alternate diplomas for some or all students and/or by funding remediation programs for students who struggle to pass exit exams" (p. 1).

McNeil, N. M., Grandau, L., Knuth, E. J., Alibali, M. W., Stephens, A. C., Hattikudur, S., & Krill, D. E. (2006). Middle-school students' understanding of the equal sign: The books they read can't help. Cognition and Instruction, 24(3), 367–385. Retrieved from http://www3.nd.edu/~nmcneil/McNeiletal06.pdf

"This study examined how 4 middle school textbook series (2 skills-based, 2 standards-based) present equal signs. Equal signs were often presented in standard operations-equals-answer contexts (e.g., 3 + 4 = 7) and were rarely presented in nonstandard operations on both sides contexts (e.g., 3 + 4 = 5 + 2). They were, however, presented in other nonstandard contexts (e.g., 7 = 7). Two follow-up experiments showed that students’ interpretations of the equal sign depend on the context. The other nonstandard contexts were better than the operations-equals-answer context at eliciting a relational understanding of the equal sign, but the operations on both sides context was best. Results suggest that textbooks rarely present equal signs in contexts most likely to elicit a relational interpretation—an interpretation critical to success in algebra" (p. 367).

The four middle-school textbook series (Grades 6 to 8) analyzed included:

According to McNeil et al. (2006), "all four textbook series focus on both skills and concepts to some degree; however, [they classified] for the sake of comparison and conciseness" (p. 372).

The caveat to this study is that results might only apply to the four textbook series for grades 6-8 examined.  Educators should examine their own texts for how the equal sign is used.

Morgan, P., Farkas, G., & Maczuga, S. (2014, June 25). Which instructional strategies most help first-grade students with and without mathematics difficulties? Education Evaluation and Policy Analysis, 20(10), 1-22.  DOI: 10.3102/0162373714536608. Retrieved from http://epa.sagepub.com/content/early/2014/06/20/0162373714536608.full.pdf

A major finding from this study is that "First-grade teachers in the United States may need to increase their use of teacher-directed instruction if they are to raise the mathematics achievement of students with MD [mathematics difficulties]."  Such teacher-directed instruction includes "routine practice and drill."  Co-author Paul Morgan discusses major findings from this study in a video posted on YouTube.

National Council of Teachers of Mathematics. (2012, February). Large scale mathematics assessments and high-stakes decisions. Retrieved from http://www.nctm.org/Standards-and-Positions/Position-Statements/Large-Scale-Mathematics-Assessments-and-High-Stakes-Decisions/

NCTM has a section on its web site for standards and positions.  With regard to large scale mathematics assessments and high-stakes decisions, NCTM stated:

Large-scale mathematics assessments are only one of a variety of measures that should be used when making high-stakes decisions that significantly impact schools and students. Such decisions should also take into account relevant and valid data on classroom-based performance, derived from assessments, both formative and summative, that offer students a range of opportunities to demonstrate their mathematical knowledge. Moreover, such decisions must consider the resources provided to support high-quality mathematics instruction as well as the opportunities afforded to students to meet the standards being assessed. When significant, high-stakes decisions are based on a single large-scale mathematics assessment, such as a state mandated end-of-year test, the likelihood of error stemming from the validity of inferences made from just one data point is higher.  (para. 1)

Rittle-Johnson, B., & Jordan, N. C. (2016). Synthesis of IES-Funded Research on Mathematics: 2002–2013 (NCER 2016-2003). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.  Retrieved from http://ies.ed.gov/ncer/pubs/20162003/pdf/20162003.pdf

This report from the IES is highly relevant to math educators.  It lists 28 ways that federally funded research during this time period contributed to what we know on how to teach mathematics and approaches to professional development.  Almost 200 federally funded studies about math learning and teaching were analyzed.  The report is organized into two sections: 1.  Whole Numbers, Operations, and Word Problem Solving in Elementary School (10 contributions), and 2. Fractions and Algebra in Middle School (14 contributions).  The report revealed 4 contributions related to professional development approaches.

Rosen, L. D. (2010). Rewired: Understanding the iGeneration and how they learn.  Palgrave Macmillan.  Available from http://us.macmillan.com/rewired/LarryDRosen

Rosen's book contains nine chapters and 256 pages:

  1. Why tweens and teens hate school
  2. Welcome to the iGeneration
  3. An explosion of WMDs : wireless mobile devices
  4. Multitasking madness
  5. Real life or screen life? : the educational opportunities of immersive social networking and virtual worlds
  6. Tapping into a very creative generation of students
  7. Media literacy among 21st-century kids
  8. Concerns, worries, and barriers
  9. Rewiring education

Rosen suggested that teachers should begin to use cell phones as tools for mobile learning.  Students should generate original content online as part of learning (e.g., via setting up a MySpace or Facebook page).  They should be taught how to judge the trustworthiness of media sources and use the internet to help provide a global perspective.  Teachers need to teach media literacy so that learners are not just gathering superficial information, but using it to gain deeper understanding.

Rudner, L., & Schafer, W. (Eds.). (2002). What teachers need to know about assessment. Washington, D.C.: National Education Association. Retrieved from http://math.nie.edu.sg/pgde/downloads/teachers.pdf

This 111-page book in pdf format is available free online. You will learn fundamental concepts related to all assessments; essential classroom assessment concepts; and useful concepts and issues pertaining to district, state, and national assessments.  Teachers will learn how to construct multiple choice and performance assessments, how to construct and evaluate scoring rubrics, and read about essential skills for students.  This latter includes improving the quality of student notes, how to study for tests, and how to avoid the traps associated with standardized tests.

Scherer, M. (2001, September). How and why standards can improve student achievement: A conversation with Robert J. Marzano. Educational Leadership, 59(1), 14-18. Retrieved from http://www.ascd.org/publications/educational_leadership/sept01/vol59/num01/toc.aspx

In an era of accountability that has been created by technology and the information explosion, we must be specific about what students must know and be able to do. Marge Scherer, Editor in Chief of Educational Leadership, posed several key questions on how and why standards can improve student achievement in a conversation with Robert J. Marzano, Senior Fellow at the Midcontinent Research for Education and Learning Institute (McREL, http://www.mcrel.org). Marzano's comments fell into three categories: a manageable number of standards, a change in our record-keeping system that does not increase teacher clerical work, and the need for a repertoire of instructional strategies. Highlights follow.

Step one toward implementing standards is to cut the amount of content addressed within standards by about two-thirds. The sheer number of those standards is the biggest impediment to their implementation. Someone at the district or school level has to cut down the content to essentials related to standards. This would give teachers ample time to cover the essential knowledge in the time allotted and provide them with room to supplement that content. Schools and teachers are looking to state departments of education for guidance.

In addition to trimming standards, a monitoring system is needed that allows tracking of student progress on specific standards. Marzano suggested a change from our grading practices to standards-based grading. This would require a change in record keeping and the use of rubric scores or percentage scores on specific standards that were covered in a course. Administrators have to set up a record-keeping and monitoring system that is easy for teachers to use, if they expect teachers to implement standards. Over time, you would plot student progress on specific standards. Those patterns are more reliable and valid than a single score from a year-end test. Getting feedback on student progress as often as possible, at least once a year, is absolutely essential to the teaching and learning process.

Research at McREL identified classroom practices that generally lead to achievement gains. These include:

Schoenfeld, A. H. (2002). Making mathematics work for all children: Issues of standards, testing, and equity. Educational Researcher, 31(1), 13-25. Retrieved from http://gse.berkeley.edu/sites/default/files/users/alan-h.-schoenfeld/Schoenfeld_2002%20Making%20Math%20Work%20ER.pdf

Schoenfeld discussed four conditions necessary for providing high quality mathematics instruction for all students: high quality curriculum; a stable, knowledgeable, and professional teaching community; high quality assessment that is aligned with curricular goals; and stability and mechanisms for the evolution of curricula, assessment and professional development.

It has taken time for curricula to be developed and implemented that is aligned to 1989 NCTM Principles and Standards, and more time for hard data based on large-scale implementations to be collected on their effectiveness.  In terms of the current state, Schoenfeld stated that the body of data indicates the following (p. 16):

  1. "On tests of basic skills, there are no significant performance differences between students who learn from traditional or reform curricula.
  2. On tests of conceptual understanding and problem solving, students who learn from reform curricula consistently outperform students who learn from traditional curricula by a wide margin.
  3. There is some encouraging evidence that reform curricula can narrow the performance gap between Whites and under-represented minorities."

Schoenfeld presented data from Pittsburgh Public Schools to illustrate effectiveness of reform curricula, and noted reform experiences in Michigan and Massachusetts.

He stated that "teaching is a profession more in name than in reality", which is "a national outrage and a national pathology" (p. 16).  The reason for this stems from the need for more training, the typical expectation being that "one year of teacher training will prepare candidate teachers to take on full responsibilities of the classroom" and that once in the field "the vast majority of teachers have minimal opportunities for professional growth" (p. 16).

There are at least two large-scale assessments aligned with NCTM Principles and Standards.  These are New Standards Mathematics Reference Examination and a standards-based assessment developed by the Mathematics Assessment Resource Service (MARS).  See:

Siegler, R., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., Thompson, L.m & Wray, J. (2010, September). Developing effective fractions instruction for kindergarten through 8th grade: A practice guide (NCEE #2010-4039). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.  Retrieved from http://ies.ed.gov/ncee/wwc/PracticeGuide/15

This practice guide is intended for teachers in K-8 to help their learners gain proficiency in understanding and problem-solving with fractions.  "This document uses the term fractions rather than rational numbers. The term fractions refers to the full range of ways of expressing rational numbers, including decimals, percentages, and negative fractions" (p. 7).  The following five recommendations are included.  Each is accompanied by examples on how to carry out the recommendation:

  1. Build on students’ informal understanding of sharing and proportionality to develop initial fraction concepts.
  2. Help students recognize that fractions are numbers and that they expand the number system beyond whole numbers. Use number lines as a central representational tool in teaching this and other fraction concepts from the early grades onward.
  3. Help students understand why procedures for computations with fractions make sense.
  4. Develop students’ conceptual understanding of strategies for solving ratio, rate, and proportion problems before exposing them to cross-multiplication as a procedure to use to solve such problems.
  5. Professional development programs should place a high priority on improving teachers’ understanding of fractions and of how to teach them.

Stepanek, J. (2000). Mathematics and science classrooms: Building a community of learners. It's just good teaching. Northwest Regional Education Laboratory Math and Science Center. Retrieved from http://www.nwrel.org/msec/just_good/10/titlepg.html

Stepanek has several chapters, but among them are three models for collaborative learning (democratic, caring, and ecological) and group process skills and strategies to eliminate stratification in groups.

Trends in International Mathematics and Science Study (TIMSS, formerly the Third International Mathematics and Science Study). TIMSS information is located at the National Center for Educational Statistics.  For math, there is a sampling of questions (Dare to Compare) that appeared on prior TIMSS and National Assessment of Education Progress Exams: http://nces.ed.gov/timss/.

Walters, K., Smith, T., Leinwand, S., Surr, W., Stein, A., & Bailey, P. (2014, November). An up-close look at student-centered math teaching: A study of highly regarded high school teachers and their students. Quincy, MA: Nellie Mae Education Foundation.  Retrieved from http://www.nmefoundation.org/resources/student-centered-learning/an-up-close-look-at-student-centered-math-teaching

This case study of 22 highly regarded high school teachers and their students in six New England states and New York was conducted by researchers from The American Institutes of Research (AIR), with support from the Nellie Mae Education Foundation.  The team examined student-centered math instruction to "provide rich descriptions of how the approach [played] out in several classrooms, taking into account how teachers’ personal philosophy and the school’s instructional context might influence their practice. The case study also provided insights into students’ perspectives on different approaches to mathematics instruction. Second, the researchers [looked] across a larger sample of classrooms to determine the effects of varying degrees of student-centeredness on students’ engagement with learning and their problem-solving skills" (Executive summary, p. 3).

The researchers developed and then applied an analytic framework with four types of learning opportunities in mathematics. In student-centered mathematics learning environments, students have meaningful opportunities to:

Data sources included videos of lessons, weekly instructional logs, teacher interviews, and student focus groups.

Key findings on student outcomes:

As a summary of student-centered instructional practices of teachers, findings were reported on orchestration of discussion and instructional tasks, as follows.

Orchestration of discussion:

Instructional tasks:

Two conclusions stand out among those provided: "Highly abstract mathematical concepts can be presented in student-centered ways, with positive outcomes for students."  And, "Teaching philosophy and instructional context may affect how strongly and consistently teachers enact student-centered approaches." (Executive summary, p. 7).

Wagner, T. (2003, November 12). Beyond testing: The 7 disciplines for strengthening instruction. Education Week, 23(11), 28, 30. Retrieved from http://www.tonywagner.com/resources/beyond-testing

Tony Wagner, co-director of the Change Leadership Group (CLG) at Harvard University's graduate school of education, reported on strategies used for improving teaching in districts that have dramatically raised the level of student achievement for the lowest quartile of students, including those from the most at-risk populations. He discussed seven practices, which the CLG identified, that appear to be central to any successful instructional-improvement effort.  All of these might not be implemented at once, some must come before others, but none can be skipped.  Quoting his words:

Wenglinsky, H. (2002, February 13). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10(12). Retrieved from http://epaa.asu.edu/ojs/index.php/epaa/issue/view/10

Wenglinsky found that teacher classroom practices have a significant effect on student achievement. Additionally, high-quality professional development focusing on higher-order thinking skills and diversity issues does appear to strongly influence classroom practice. Teacher quality and classroom practice can have an effect on student achievement equal to or exceeding that of socioeconomic status (SES) of students.

In addition, he noted aspects of teacher quality that are related to student achievement when class size and SES are taken into account. In particular, the following five variables are positively associated with achievement:

Williams, T., Kirst, M., Haertel, E., et al. (2010). Gaining Ground in the Middle Grades: Why Some Schools Do Better. Mountain View, CA: EdSource. Retrieved from http://www.edsource.org/middle-grades-study.html

Williams, Kirst, Haertel, et al. from EdSource and Stanford University conducted survey research in the 2008-09 school year, which involved 303 middle grades schools in California. The team surveyed 303 principals, 3752 English Language Arts (ELA) and math teachers in grades 6-8, and 157 superintendents of the districts and charter management organizations that oversee the schools.  They analyzed the reported district and school practices, correlating them with California's spring 2009 standardized test results in ELA and math in grades 6, 7, and 8 taken by approximately 204,000 students in their sample.  The study is significant for documenting the range of traditional and newer policies and practices that are in place in those middle schools and for identifying which of those practices and policies differentiated higher-performing schools from lower-performing schools with similar student populations on standards-based tests.  Findings have implications for practices that might be incorporated in other settings, which lend themselves for achievement and better standardized testing results independent of student background.

The following are some of the major findings noted in the Narrative Summary of this study:

Woodward, J., Beckmann, S., Driscoll, M., Franke, M., Herzig, P., Jitendra, A., Koedinger, K. R., & Ogbuehi, P. (2012). Improving mathematical problem solving in grades 4 through 8: A practice guide (NCEE 2012-4055). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/PracticeGuide/16

Woodward and colleagues provided five recommendations for improving mathematical problems solving in grades 4-8 and discussion of the research supporting the following:

  1. Prepare problems and use them in whole-class instruction.
  2. Assist students in monitoring and reflecting on the problem-solving process.
  3. Teach students how to use visual representations.
  4. Expose students to multiple problem-solving strategies.
  5. Help students recognize and articulate mathematical concepts and notation.

Recommendations 2 and 3 are supported by "strong" evidence, according to research reviewed for this practice guide.

 

Back to top

 

Technology Integration

 

Learn More...

Students in a computer labTechnology enhanced learning activities should be aligned with local, state, and national standards, should include well-defined assessments, should be accessible to all learners (including those with disabilities), and should contain multiple learning strategies.  Strategies for technology integration that might be used include active learning strategies, constructivist learning strategies, cooperative learning strategies, authentic learning strategies, and intentional/reflective learning strategies.

Read more about Good Models of Teaching with Technology, a publication of the Education Alliance at Brown University.

Orey, M.(Ed.). (2001). Emerging perspectives on learning, teaching, and technology. Retrieved from http://epltt.coe.uga.edu/index.php  This book is freely available online with articles, videos, animations, narrations, and images on learning and cognitive theories, instructional theories and models, inquiry and direct instruction strategies, and more.  It's continually updated.  You'll also find discussion on technology tools for teaching and learning.  Highly recommended.

TechMatrix.org maintains a searchable database for math, reading, writing, and assistive technologies.  The section on Research includes multiple articles on theory and practice for using technology to improve learning.  For example, there are over 40 research articles on teaching math using technology in various forms.

 

Battista, M. T. (1998). Computer environments that engender students’ construction of mathematical ideas and reasoning: A constructivist perspective. Paper presented at the ENC Technology and NCTM Standards 2000 Conference. Arlington VA, June 5-6, 1998.  Retrieved from http://mathforum.org/technology/papers/papers/battista/battista.html

Carrol, T. G. (2000). If we didn't have the schools we have today, would we create the schools we have today? Contemporary Issues in Technology and Teacher Education, 1(1), 117-140. Retrieved from http://www.citejournal.org/publication/volume-1/issue-1-00/

Carroll included discussion of the classroom of tomorrow and six reasons why the web has won.

Cheung, A., & Slavin, R. E. (2011, July). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.  Retrieved from http://www.bestevidence.org/math/tech/tech_math.htm

Cheung and Slavin included "74 qualifying studies, with a total sample size of 56,886 K-12 students" in this meta-analysis.  Three categories of educational technology were reviewed: Computer managed learning (via the program Accelerated Math); Comprehensive models that included both computer assisted instruction (CAI, e.g., via programs Cognitive Tutor and I Can Learn) and non-CAI activities in math instruction; and Supplemental CAI technology for individualized CAI to supplement traditional instruction (e.g., via programs Jostens, PLATO, Larson Pre-Algebra, and SRA Drill and Practice).  "Findings of the review indicate that educational technology applications produce a positive but small effect (ES=+0.16) on mathematics achievement. In particular, supplemental CAI had the largest effect, with an effect size of +0.19. The other two categories, computer-managed learning and comprehensive models, had a much smaller effect size, +0.09 and +0.06, respectively."

 

Learn more on how to interpret effect sizes.

Friendly reminder Gif

"An effect size indicates the number of standard deviations by which the treatment group mean differs from the control group mean on the outcome measure of interest" (Morrison, Morrison, & Ross, 2016, p. 5).  In other words, it's a way to quantify the size of the difference between two groups:

Effect size = ( [Mean of the experimental group] - [Mean of the control group] ) / Standard Deviation

In Effect Size Matters in Educational Research Robert Slavin (2013) commented that it is difficult to decide if a program is worth attempting in your own setting based on effect sizes reported in results of experimental research.  Studies vary in quality and in their nature.  According to Slavin:

"... many features of studies give hugely inflated estimates of effect sizes. In order of likely importance, here are some factors to watch for:

  • Use of measures made by the researchers.
  • Very brief studies (often, one hour or less)
  • Studies with small sample sizes

Studies that incorporate any of these elements can easily produce effect sizes of +1.00 or more. Such studies should be disregarded by readers serious about knowing what works in real classrooms and what does not" (para. 4).

The Institute of Education Sciences published a report by Mark Lipsey and colleagues (2012) on how to interpret effect sizes: Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms.  While studies might report significant results, practical significance (is implementation worth it) is another issue, which is also addressed.  According to Lipsey et al. (2012):

"Practical significance is not an inherent characteristic of the numbers and statistics that result from intervention research—it is something that must be judged in some context of application. To interpret the practical significance of an intervention effect, therefore, it is necessary to invoke an appropriate frame of reference external to its statistical representation. We must have benchmarks that mark off degrees of recognized practical or substantive significance against which we can assess the intervention effect" (p. 26).

For example, in relation to student achievement, some useful benchmarks might consider how large the size of the effect is if you think about it in terms of the following:

  • "what it might add to a year of average academic growth for the target population of students"
  • "its ability to narrow a policy-relevant gap in student performance"
  • its "relation to what prior interventions have been able to accomplish"
  • "Do the benefits of a given intervention outweigh its costs?" (p. 26)

Slavin (2016a) further commented on effect sizes in his blog post, What is a Large Effect Size?  In posing the question "When are they big enough to matter for practice?" (para. 5), he indicated "The answer turns out to mainly depend on just two factors: Sample size, and whether or not students, classes/teachers, or schools were randomly assigned (or assigned by matching) to treatment and control groups" (para. 6).

There are other factors to consider "beyond effect size or statistical significance in adopting a program to improve education outcomes" (Slavin, 2016b, para. 2).  Staff might also consider the following factors:

  • Cost-effectiveness
  • Evidence from similar schools
  • Immediate and long-term payoffs
  • Sustainability
  • Breadth of impact
  • Low-hanging fruit (i.e. per Slavin, "interventions that so inexpensive or easy to adopt and implement that it would be foolish not to do so.")
  • Comprehensiveness (Slavin, 2016b, para. 2).

For a full report, see:

Cheung, A., & Slavin, R.E. (2015, September). How methodological features affect effect sizes in education. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.  Retrieved from http://www.bestevidence.org/methods/methods.html

 

Johnson, J., & Toms Barker, L. (eds.) (2002).  Assessing the impact of technology in teaching and learning: A sourcebook for evaluators.  Ann Arbor, MI: Institute for Social Research at the University of Michigan. Retrieved from http://rcgd.isr.umich.edu/tlt/TechSbk.pdf

This 184 page sourcebook, available for free in PDF format online, provides an overview of measurement issues in seven areas as well as examples of measures used in current projects.  It would be of value to evaluators who are assessing the role of technology in American education.  The areas include learner outcomes in the cognitive and affective domains and in adult education, teacher outcomes related to changed pedagogy and improving technology skills, technology integration, and disseminating the lessons of technology projects.  The first chapter on learner outcomes in the cognitive domain, for example, includes the merits and difficulties of using standardized tests, tailored tests, and authentic assessments in the evaluation of educational projects.

Kerrey, B., & Isakson, J. (2000, December 19).  The power of the Internet for learning: Moving from promise to practice. Washington, DC: Web-Based Technology Commission. Available as archived information: http://www.ed.gov/offices/AC/WBEC/FinalReport/index.html

This final report of the Web-based Technology Commission contains over 100 pages and was presented to the President and the Congress of the United States in December, 2000. Among the contents are discussions of access to broadband technologies; professional development and how technology can enhance teaching; correcting the paucity of research and development; online content; removing regulatory restrictions to e-learning; privacy, protection, and safety; funding for e-learning, and the call for national action.

Kleiman, G. M. (2000, April-June). Myths and realities about technology in K-12 schools. Leadership & the New Technologies, 14. Retrieved from http://www.edtechleaders.org/documents/myths.pdf

Glenn Kleiman of The Center for Online Professional Education discussed the realities surrounding five myths that relate to computer availability in schools, goals and best practices for computer use in classrooms, teacher implementation stages for effective use, district technology plans, and equity and the digital divide.  The message is that short term solutions will not work.  The key is not how many computers are available, but, in his words, "how we define educational visions, prepare and support teachers, design curriculum, address issues of equity, and respond to the rapidly changing world" [Online].

Ray, B. (2005). Mining what we know about handheld computers: A review of the [anecdotal] evidence. The Turkish Online Journal of Educational Technology, 4(2), article 1. Retrieved from http://www.tojet.net/articles/v4i2/421.pdf

According to Beverly Ray (2005) who reviewed the literature on handheld computers, "The empirical evidence suggests that the integration of handheld technology into the K-12 classrooms promotes 1) teacher productivity and 2) student-centered learning. However, despite a wealth of empirical and anecdotal evidence there is no research base to support these assertions. Further research supporting their effectiveness, however, remains to be done" (p. 5).

Shields, J., & Poftak, A. (2002). A report card on handheld computing.  Technology & Learning, 22(7), 24-36.

Jean Shields and Amy Poftak discussed the pros and cons of handheld computers in K-12.  They present a short history of these small devices for one-on-one computing in schools, their potential for learning, and explore applications for integrating handhelds into instruction.  They reference actual classrooms where handhelds are used and include a list of links to resources to learn more.

Wallace, A. (2013, October 25). CIPA: 10 years later, there is still confusion. TechLearning Magazine: Features.  Retrieved from http://www.techlearning.com/features/0039/cipa-10-years-later-there-is-still-confusion/54343

Andrew Wallace tackled the confusion that still exists among those who are required to interpret the Child Internet Protection Act (CIPA) and implement it.  Stating CIPA requirements in a nutshell:

School Districts must:

Wells, J., & Lewis, L. (2006, November).  Internet access in U.S. public schools and classrooms: 1994-2005 (NCES 2007020). U.S. Department of Education. Washington, DC: National Center for Education Statistics.  Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007020

John Wells and Laurie Lewis presented survey findings (no survey in 2004) on school connnectivity, student access to computers and the Internet, technologies and procedures to prevent student access to inappropriate material on the Internet, teacher professional development on how to integrate the use of the Internet into the curriculum, and use of the Internet to provide opportunities and information for teaching and learning.

Weston, M. E. & Bain, A. (2010). The end of techno-critique: The naked truth about 1:1 laptop initiatives and educational change. Journal of Technology, Learning, and Assessment, 9(6). Retrieved from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1611

 Weston and Bain analyze and respond to "a generation of criticism leveled at 1:1 laptop computer initiatives."  They present "a review of the key themes of that criticism and [offer] suggestions for reframing the conversation about 1:1 computing among advocates and critics.  Efforts at changing, innovating, and reforming education provide the context for reframing the conversation.  Within that context, [they] raise questions about what classrooms and schools need to look and be like in order to realize the advantages of 1:1 computing.  In doing so, [they] present a theoretical vision of self-organizing schools in which laptop computers or other such devices are essential tools" (Abstract section).

Willard, N. (2002). Keeping kids safe online. Education World. Retrieved from http://www.educationworld.com/a_tech/tech119.shtml

The Child Internet Protection Act requires districts to monitor student use of the Internet and to implement technology-based measures to protect against student access to online content that may be harmful to minors. Willard noted four core components for a comprehensive plan to address online safety and Internet use by districts:

July 24, 2007 Note: Willard also noted that districts that want to have blocking in place should consider the use of the Internet Content Rating Association system.  ICRA is now part of the Family Online Safety Institute.

 

Back to top

 

A row of open books Gif

Continue your own research to implement standards-based instruction.

Need help? NoodleTools is a suite of interactive tools designed to aid students and professionals with their online research. Get the help you need to select a search engine, find some relevant sources, and cite those sources in MLA or APA style.  http://www.noodletools.com/

Education Commission of the States maintains a database of readings on nearly any education issue of interest, such as accountability, assessment, closing the achievement gap, curriculum, distance education, mathematics, standards, technology.  Visit http://www.ecs.org

Read these books online for free:

  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.) (2000).  How people learn: Brain, mind, experience, and school: Expanded edition. Washington, DC: The National Academies Press.

    This book is the expanded edition of the 1999 work by these authors.  It has four parts.  Part I, the Introduction, discusses learning from speculation to science.  Part II is devoted to learners and learning: how experts differ from novices, learning and transfer, how children learn, and mind and brain.  Part III delves into teachers and teaching: the design of learning environments, effective teaching examples in history, mathematics, and science, teacher learning, and technology for learning.  Part IV includes future directions for the science of learning with conclusions and recommendations for research.

  • Center for Education. (2001). Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education. Washington, DC: The National Academies Press.

  • Hamilton, L. S., Stecher, B. M., & Klein, S. P. (eds.). (2002). Making sense of test-based accountability in education. Santa Monica, CA: Rand Corporation.

    The authors discussed high-stakes testing and offer recommendations for more-effective test-based accountability systems.  Content includes, for example, historical perspective of high-stakes tests and their use today, aligning tests with standards, technical criteria for evaluating tests, consequences of high-stakes testing on school and classroom practice, and the political view of accountability.

  • Lewis, A. C. (1999). Figuring it out: Standards based reforms in urban middle grades. New York, NY: Edna McConnell Clark Foundation. ERIC document ED439165.

    Exemplary content includes how to design standards-based classrooms; the difference that standards make for students, teachers, and principals; what holds back standards-based reforms, and what reforms need to continue.

  • National Research Council. (2005). How students learn: History, science, and mathematics in the classroom. Committee on How People Learn, a targeted report for teachers, M. S. Donovan and J. D. Bransford, Editors. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

    This book has its roots in the book How people learn: Brain, mind, experience, and school (noted above).  Its goal is to present "examples of how the principles and findings on learning can be used to guide the teaching of a set of topics that commonly appear in the K-12 curriculum. ... Each area is treated at three levels: elementary, middle, and high school" (p. vii).

  • Ball, D. L. (2003). Mathematical proficiency for all students: Toward a strategic research and development program in mathematics education. Santa Monica, CA: Rand Corporation.

    The RAND Mathematics Study Panel with Deborah Loewenberg Ball, Chair, proposed three areas for research that should be emphasized to improve math education:

    • Development of the mathematical knowledge that teachers need to effectively teach students from diverse backgrounds.
    • Teaching and learning required for mathematical thinking and problem solving.
    • Teaching and learning algebra from kindergarten through 12th grade, because algebra is central to proficiency in mathematics.

 

References

Cheung, A., & Slavin, R.E. (2015, September). How methodological features affect effect sizes in education. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.  Retrieved from http://www.bestevidence.org/methods/methods.html

Lipsey, M., Puzio, K., Yun, C., Hebert, M., Steinka-Fry, K., Cole, M., et al. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. Washington, DC: Institute of Education Sciences. Retrieved from http://ies.ed.gov/ncser/pubs/20133000/ 

Morrison, G., Morrison, J., & Ross, S. (2016, March). A review of the research literature on the infusion of technology into the school curriculum. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.

Slavin, R. (2016a, March 10). What is a large effect size? [Web log post]. Retrieved from http://www.huffingtonpost.com/robert-e-slavin/what-is-a-large-effect-si_b_9426372.html

Slavin, R. (2016b, May 26). How much difference does an education program make? [Web log post]. Retrieved from http://www.huffingtonpost.com/robert-e-slavin/how-much-difference-does_b_10143048.html

Slavin, R. (2013, January 9). Effect size matters in educational research [Web log post]. Retrieved from http://blogs.edweek.org/edweek/sputnik/2013/01/effect_size_matters_in_educational_research.html

 

Black line

Back to top Education Research: Page 1  |  2  | 

Binoculars GifGo to Related Topic:  State and National Education Standards and The Best Rated Standards Resources