Authors’ Note
Using data to meet students’ individual learning needs is a common practice in U.S. K–12 education systems. In recent years, it has become more common for schools to expand the adoption of data-use practices by drawing on detailed data to inform an understanding of individual students’ learning needs, interests, and preferences and using that information to design highly tailored instructional experiences. Often designated as personalized learning, these models seek to enable what and how a student learns on a daily basis to be:
- driven by frequent receipt and use of data to diagnose and inform the individual student’s needs, interests, and context;
- unconstrained by the needs of other students or by external requirements for grade-level content coverage; and
- informed by ongoing conversations with the student and the adults in his or her life.
This article is a reprint of the executive summary of our larger report, which describes findings from a three-year study of schools striving to use this instructional approach schoolwide. The study examined the implementation of personalized learning approaches as well as student achievement outcomes in those schools. Achievement analyses found that students made gains in mathematics and reading that were significantly greater than those of a matched comparison group. Moreover, the students started out mostly performing below the national averages in mathematics and reading, and generally ended with scores near or above the national averages after two years. Readers of this special issue may be particularly interested in the study’s findings regarding how data supported personalized learning approaches and may have helped lead to these positive achievement results. Those findings are summarized below, but there is considerably more detail in the full report, which can be accessed here.
Information on data use is most concentrated in the Learner Profiles section of the report, but is also touched on in the sections on Personal Learning Paths, Competency-Based Progress, and Flexible Learning Environments. To briefly summarize, the approaches adopted by schools in the study relied in many ways on the frequent receipt and use of data. Teachers reported needing and using a broad variety of data sources to inform instructional decisions, and administrators reported that their schools used a similarly broad variety of data to track and understand student progress. Two-thirds of administrators reported sharing students’ data with them and engaging them in discussions about the data.
In a section titled Relating Implementation to Outcomes the study found that schools with the largest achievement gains were the only ones that implemented three features of personalized learning in tandem. It is notable that two of those features are related to data use: (1) using student grouping strategies that are driven by data and responsive to student needs; and (2) providing data to students and including them in discussions about how the data relate to the students’ personal learning goals.
Executive Summary
The Bill & Melinda Gates Foundation has engaged RAND to carry out an ongoing study of foundation-funded schools that are employing promising approaches to personalized learning. This research is part of the foundation’s public commitment to spread effective practices across districts and charter networks, develop innovative roles for teachers, and support implementation of college-ready standards. This is the second report in a series focused on the achievement data, school design characteristics, and teacher and student perceptions of schools that are implementing personalized learning. The achievement findings in this report focus on 62 public charter and district schools that are pursuing a variety of personalized learning practices. In a smaller set of 32 schools, the report examines details of personalized learning implementation and the relationship of implementation to outcomes.
The concept of personalized learning has been around for some time, but the adoption of personalized learning approaches has increased significantly in recent years due in part to rapid advances in technology platforms and digital content. Although there is not yet one shared definition of personalized learning, leading practitioners in the field generally look for the following: (1) systems and approaches that accelerate and deepen student learning by tailoring instruction to each student’s individual needs, skills, and interests; (2) a variety of rich learning experiences that collectively prepare students for success in the college and career of their choice; and (3) teachers’ integral role in student learning: designing and managing the learning environment, leading instruction, and providing students with expert guidance and support to help them take increasing ownership of their learning.
Although these core principles are common among the schools in the study, there is considerable diversity in the details of the schools’ instructional models because innovation was encouraged in the competitive grant programs they participated in. Each school received funding after rigorous evaluation of its leadership team and instructional vision as part of at least one of the following three Gates foundation-supported initiatives: Next Generation Learning Challenges (NGLC), Charter School Growth Fund’s Next Generation School Investments, and the Gates Foundation’s Personalized Learning Pilots. (See the full report for more detailed descriptions of these initiatives.)
Student Achievement Results
Students made gains in mathematics and reading that were significantly greater than a comparison group made up of similar students selected from comparable schools. These results are consistent with the findings reported last year but with a sample nearly three times as large. The gains in both mathematics and reading at schools that have implemented personalized learning approaches were relatively large compared with gains in studies of schools with other types of interventions. A majority of the schools had statistically positive results. Importantly, although students started out mostly performing below the national averages in mathematics and reading, they generally ended with scores near or above the national averages after two years in personalized learning schools. These results suggest that the effects of personalized learning on student achievement are promising.
School Design Characteristics
The implementation findings rely on data collected in schools implementing personalized learning with funding from the NGLC program. There are 32 such schools that also administered the Northwest Evaluation Association’s Measures of Academic Progress mathematics and reading assessments during the 2014–15 school year. The schools are located predominantly in urban areas with the exception of two rural schools. They tend to serve large proportions of minority students from low-income families. The school-level median of students of color is 75 percent, and the school-level median of students eligible for free or reduced-price lunch is 80 percent.
Although the schools have considerable diversity in their models of personalized learning, the Gates foundation, along with other funders and leaders in the personalized learning space, identified five strategies that are often present in the schools. As the following descriptions suggest, each strategy encompasses a set of tools and features of the personalized learning environment. Some of these, such as the provision of flexible pathways, are central to a personalized approach, whereas others (e.g., use of technology) might be viewed more as enablers of personalized learning. We find this framework to be a useful way to organize discussion of school design features and implementation.
LEARNER PROFILES give teachers an up-to-date record that provides a deep understanding of each student’s individual strengths, needs, motivations, progress, and goals to help inform his or her learning. Teachers work with students to understand their data and create individual goals. Data from multiple sources (e.g., projects, tests, quizzes, presentations, software, or non-cognitive factors) are used to understand student progress.
Teachers reported using a variety of data and other resources to inform their instructional decisions, and all administrators reported that their schools used data from different sources to understand student progress. About half of administrators reported that their schools were developing personalized goals for students, and two-thirds were providing data to and discussing data with students.
PERSONAL LEARNING PATHS hold all students to high expectations, but the school model allows for flexibility in the path that students take through the content. Students are able to make choices about the content or structure of learning, and the school uses a variety of instructional approaches and curriculum materials to meet the learning needs of all students. In addition, there is time during the school day for one-on-one academic supports for students that are tailored to their learning needs, and there are opportunities for students to engage in meaningful learning experiences outside of school.
Administrators reported that the extent to which students were able to make choices about their learning varied by course, teacher, and age of the student. Administrators and teachers identified project-based learning approaches as one way of providing students with choice and with a personalized path through content. All schools provided time for individual academic support, which emphasized teaching developmentally appropriate content. Three-quarters of schools used a variety of instructional formats and offered out-of-school learning opportunities. Implementation of innovative, out-of-school learning opportunities was less common, and the opportunities offered were typically not technology-enabled or substantially different from traditional environments.
COMPETENCY-BASED PROGRESSION continuously assesses each student’s progress toward clearly defined goals, and assessment occurs “on demand” when a student is ready to demonstrate competency. Assessment may take a variety of forms, such as projects or presentations, as well as more traditional tests and quizzes. Students advance through the content at their own pace as they demonstrate competency.
Students’ ability to work at their own pace and advance when they had mastered the material was limited by a perceived need to emphasize teaching grade-level content. This emphasis was driven by a desire to ensure that students were progressing toward grade-level standards and external policy constraints such as standardized testing. Fewer schools seemed to be implementing competency-based progression than were implementing other personalized learning strategies.
FLEXIBLE LEARNING ENVIRONMENTS use resources such as staff, space, and time in flexible ways to support personalization. For example, classroom size, organization, or furniture does not hinder implementation of personalized learning. The structure of learning time is responsive to student needs. Student grouping strategies are flexible, responsive to student needs, and based on data. Technology is a key aspect of the school model and is available to all students.
Teachers generally reported that the learning space was supportive of personalized learning. About three-quarters of administrators reported that learning time was structured in a way that was flexible and responsive to student needs. Most schools had extended school days or school years, and they used the extra time primarily for additional mathematics or English language arts (ELA) instruction or to provide individualized support. Educators at many of the schools were thinking flexibly about how staff are used for instruction and student support. One-fifth of teachers reported holding unconventional roles such as co-teaching, job sharing, or working with small groups of students primarily under the supervision of another teacher.
EMPHASIS ON COLLEGE AND CAREER READINESS as a personalized learning strategy means that a school’s curriculum, activities, and programs develop academic and non-academic college and career readiness skills. Some examples are college visits, career surveys, career-oriented internships, college-level courses, or encouragement of college expectations. Aspects of curriculum, activities, or programs (including student advisory strategies) are intended to develop students’ skills and competencies beyond academic content. These are referred to variously as “habits of mind,” “learner identity,” “student agency,” “non-cognitive skills,” etc.
All schools incorporated ways to develop non-academic skills in preparation for life after high school into the curriculum in some way, often through advisory curricula and cooperative learning opportunities, such as group projects. Administrators of schools at all grade levels said they were developing students’ awareness of, and knowledge about, postsecondary opportunities. In schools with younger students, this generally consisted of activities such as providing information about college, talking about college, and developing a belief that college is attainable. In schools with older students, these activities typically took the form of traditional college counseling, college visits, and in some high schools, opportunities to earn college credit.
CONTEXTUAL FACTORS can influence personalized learning implementation.
High-quality supports for teaching are relevant to the implementation of all five strategies. Teachers expressed positive opinions about colleague and administrator support and about the quality and usefulness of the professional development they received. However, a majority of administrators identified teacher staffing as a challenge. This was particularly true for schools that opened in 2012. The administrators cited high staff turnover as a common problem. Site visit participants suggested that mid-year teacher departures were disruptive, particularly in new schools, which tended to have smaller staffs.
Relating Implementation to Outcomes
While no single element of personalized learning was able to discriminate between the schools with the strongest achievement outcomes and the others in the sample, we did identify three elements that, when present together, were able to do so. Among all the schools in the analysis, the most successful schools were the only ones that showed strong evidence of implementing all three: (1) student grouping strategies driven by data and responsive to student needs; (2) providing data to students and including them in discussions about how the data relate to the students’ personal learning goals; and (3) learning spaces that are supportive of, or do not hinder, implementation of other personalized learning strategies.
National Comparison of Survey Results
This study compared the teacher and student survey results from the 32 schools in the implementation analysis to results from administering nearly identical questions to a national sample of teachers and students. Grunwald Associates administered the national survey during summer 2015. The national results are intended to provide context for the findings from the personalized learning schools to help understand the ways in which the experiences of students and teachers in these schools differed from the experiences of students and teachers nationally. To facilitate this comparison, we first weighted the national survey results to more closely reflect the personalized learning sample in terms of geographic locale (e.g., urban), grade level, subject taught (by teachers), and gender (of students). However, we lacked the necessary data to include family income in the student survey weighting process and the national sample appears to be somewhat more affluent than the personalized learning sample.
The use and characteristics of learner profiles and emphasis on student choice in the personalized learning schools were similar to those in the national sample. However, teachers in personalized learning schools were more likely than those in the national sample to use technology for personalization and to report that their school’s data system is useful. Teachers in the personalized learning sample also reported greater use of instructional practices that support competency-based learning. Students in both samples agreed or strongly agreed that there was an emphasis on making them aware of instructional goals and tracking progress toward mastery. Finally, students in the personalized learning schools were more likely to report that they were able to make choices about their learning and that their mathematics and ELA instruction incorporated aspects of complex, student-centered instruction.
Conclusions
Although implementation of personalized learning varied considerably across the 32 schools studied, our findings suggest that the schools were employing a number of practices that support personalization. Teachers at most schools were using data to understand student progress and make instructional decisions, all schools offered time for individual academic support, and the use of technology for personalization was widespread. However, some strategies, such as competency-based progression, were less common and more challenging to implement. The schools that exhibited the greatest achievement growth were all implementing three elements of personalized learning—student grouping, learning spaces that support personalized learning, and opportunities for students to discuss their learning data with teachers.
The achievement findings are largely positive and promising. They indicate that compared to peers, students in schools using personalized learning practices were making greater progress over the course of two school years and that those students who started out behind were catching up to perform at or above national averages. We conducted a set of rigorous sensitivity analyses and concluded that they generally support these results. However, it is important to use caution in interpreting these results as causal effects of personalized learning. Although the analyses used the best estimation strategies possible given the nature of the data and limits to implementing a strong experimental design, it was not possible to separate personalized learning effects from other potential school effects.
RAND will produce a more comprehensive report with additional details in 2016.
Author bios
John F. Pane is Distinguished Chair in Education Innovation and Senior Scientist in RAND’s Education unit. He applies experimental and rigorous quasi-experimental methods to study the implementation and effectiveness of education innovations, with a focus on personalized learning and education technology. He received his Ph.D. in computer science from Carnegie Mellon University.
Elizabeth Steiner is a Senior Project Associate. Her research focuses on evaluating education policy interventions with an emphasis on models designed to improve equity. Some examples of her recent projects include evaluating innovative whole-school implementation models in high schools, a large study of policies designed to improve teacher and school leader effectiveness, and evaluation of personalized and competency-based learning models. She holds an M.S. in Public Policy and Management from Carnegie Mellon University.
Matthew Baird is a Full Economist at RAND whose research focuses primarily on understanding labor markets to improve outcomes for disadvantaged populations, including applications in education and health labor markets. He also is involved in education program evaluation, including for personalized learning and for principal training programs. He received his Ph.D. in economics from the University of California, Los Angeles.
Laura Hamilton is a Senior Behavioral Scientist and Associate Director of RAND Education, a faculty member at the Pardee RAND Graduate School, and an adjunct faculty member in the University of Pittsburgh’s Learning Sciences and Policy program. Her research has addressed educational assessment, accountability, the implementation of curriculum and instructional reforms, education technology, and students’ social and emotional learning. She holds a Ph.D. in educational psychology and an M.S. in statistics from Stanford University.