PREDICTIONS OF IMMINENT TRANSFORMATION are among the most reliable refrains in the history of education technology. In 1913, Thomas Edison declared that the age of books was about to give way to the age of motion pictures. He told an interviewer, “Books will soon be obsolete in the public schools. Scholars will be instructed through the eye. It is possible to teach every branch of human knowledge with the motion picture. Our school system will be completely changed inside of ten years.” When Edison’s ten-year prediction failed to come to pass, he simply gave himself more time. In 1923, speaking before the Federal Trade Commission, Edison explained, “I made an experiment with a lot of pictures to teach children chemistry. I got twelve children and asked them to write down what they had learned, from the pictures. I was amazed that such a complicated subject as chemistry was readily grasped by them to a large extent through pictures. The parts of the pictures they did not understand I did over and over again until they finally understood the entire picture. I think motion pictures have just started and it is my opinion that in 20 years children will be taught through pictures and not through textbooks.”1
One hundred years after Edison, technologists are still promising that new inventions can instantly solve challenges that education systems have faced for hundreds of years. The 2010s were a banner decade for charismatic technologists, from Knewton founder Jose Ferreira’s adaptive robot tutor in the sky to Udacity founder Sebastian Thurn’s magic formula for low-cost, global-scale learning with MOOCs. My sense at the end of the decade was that some sobriety had seeped into public conversations about the limits of learning technologies. But even at the nadir of this decade’s hype cycle, wishful thinking continued. In 2019, Dan Goldsmith, then the CEO of Instructure, the company that provides the Canvas learning management system, boasted that his company’s new learning analytics program would drive student success, make teachers more productive, and increase student retention:
We have the most comprehensive database on the educational experience in the globe. So, given that information that we have, no one else has those data assets at their fingertips to be able to develop those algorithms and predictive models. What’s even more interesting and compelling is that we can take that information, correlate it across all sorts of universities, curricula, etc., and we can start making recommendations and suggestions to the student or instructor in how they can be more successful. Watch this video, read this passage, do problems 17–34 in this textbook, spend an extra two hours on this or that.2
As someone who had spent much of the decade on MOOC research, I was taken aback by this particular claim. This prediction of a data-driven revolution in personalized learning was exactly what early MOOC advocates promised. After hundreds of millions of dollars in investments in massive courses and platforms and research across some of the world’s leading universities, nothing like what Goldsmith imagined has been accomplished. Despite the examples of the developers of Knewton, Udacity, and other technologists who had to walk back early claims of transformation, here was yet another CEO borrowing the same rhetorical tropes about how massive datasets would be transformed into revolutionary learning insights, like piles of straw spun into gold. In the years ahead, no doubt, entrepreneurs will make these same kinds of promises about artificial intelligence and virtual reality and 5G and whatever new technologies Silicon Valley unleashes upon the world. Educators should be ready.
When new education technologies fail to meet their lofty expectations, a common rhetorical move is to claim that not enough time has passed for the true effects of new technologies to be revealed. The futurist Roy Amara is credited with the claim that “we tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”3 Edison’s hundred-year-old claims about motion pictures are good test cases for this theory. If we look at learning in its broadest view, some of what Edison predicted has come to pass. Video has become a dominant medium for informal learning, from the crafters of Rainbow Loom to the creators of Minecraft and in many fields beyond. But within the complex ecologies of formal educational systems, textbooks remain central to learning experiences and video remains a supplemental resource. I suspect that in the years after 2110 when we celebrate the two-hundredth anniversary of Edison’s predictions and the hundredth anniversary of Sal Khan’s TED talk “Let’s Use Video to Reinvent Education,” educational film and video will still play a secondary role in most formal educational systems.
The rhetorical tropes of disruption and charismatic technologies center around a heroic developer creating a new technology that leads to the transformation of educational systems (Edison invents the motion picture, and textbooks are replaced in a decade by more effective instructional films). This doesn’t happen. Let me propose three essential shifts to the stories that we tell about how technology can improve learning. First, change won’t come from heroic developers or even technology firms, but from communities of educators, researchers, and designers oriented toward innovative pedagogy and a commitment to educational equity. We need villages, not heroes. Second, technology won’t transform teaching and learning. Our best hope is that technologies open up new spaces for the work of holistically improving curricula, pedagogy, instructional resources, student support, teacher professional development, policy, and other critical facets of school systems. Technology, at best, has a limited role to play in the broader work of systems change. Finally, we must let go of the hope for the kinds of dramatic shifts that sometimes do happen in consumer technologies and instead envision the work of systems change as a long process of tinkering and continuous improvement.
Promises and predictions for the transformative power of large-scale learning technologies are not going away. The task for educators in the decades ahead will be to examine new technologies as they emerge, to look past overwrought rhetoric and to dismiss egregious hype, while remaining open to possibilities for how new tools might prove useful in specific contexts, for certain subjects, or for particular students. After a century of edtech hype cycles, my dream is that educators will now have enough experience, enough data, and enough history at their disposal to defend against the next wave of overly optimistic claims by crafting their own more realistic, historically grounded predictions for the future of learning at scale.
Policymakers, administrators, teachers, and students are asked to predict the future of learning technology all the time. The principal asks, “Are there any new software subscriptions that I can buy for my teachers that would improve student learning?” The policymaker asks, “What portion of state aid for schools or universities should be designated specifically for technologies in order to improve graduation rates or retention?” The teacher asks, “Will adopting a new technology help my students learn?” The learner asks, “Is it worth spending time on this MOOC, and what will the certificate be worth years later?”
When encountering a new large-scale learning technology, I have found four questions particularly useful in situating a new product in the long history of education technology:
Claims of novelty are central to the charismatic rhetoric of technology evangelists. Canvas’s Goldsmith claimed that “no one else” had the data assets that they do, and thus their unique data resources would usher in a new era of personalized learning. The question “What’s actually new here?” invites comparisons to related efforts and a skeptical orientation. MOOC researchers have similarly massive datasets, which have proven useful for some policy insights but have not enabled breakthrough research in personalized learning, despite extensive efforts. The Pittsburgh Science of Learning Center’s Datashop has reams of data on adaptive tutors; there are researchers using huge datasets from Scratch, Khan Academy, and all kinds of learning management systems. Strictly speaking, it is absolutely true that Canvas is the only company with the exact data assets of Canvas, but plenty of other large datasets of online learning behavior exist. If these older datasets haven’t led to a sea change in personalized learning, it is reasonable to expect that Canvas’s new data won’t either.
Even though edtech evangelists often seem unaware of the long history of education technology, I have found history to be a reliable and useful guide for predicting the future of learning at scale. If you can figure out where a new technology fits in the genealogy of large-scale learning technologies, you have a good chance of predicting how a new technology will operate in schools based on the track records of the ancestors of that new technology. Armed with this understanding, you can then probe the potential value of incremental contributions: Is there anything about Canvas’s dataset that differs from prior datasets used for educational data science that might lead to some incremental advance in the field? Answering probing questions like these requires situating new entrants in the long history of education technology.
Most new large-scale learning technologies fit reasonably well into one of the three genres of learning at scale that I described in the first half of this book. After asking, “What’s new?,” a trio of questions about sequencing, pedagogy, and technology should follow. Ask about who guides the learner’s sequence of actions, what pedagogical traditions are enacted in the learning activities, and what technologies are used to engage the learner.
If an instructor determines the sequence of learning for students, if the pedagogy appears instructionist (with experts directly transmitting new knowledge to learners), and if the technology is a combination of a learning management system combined with autograders to assess and track learner progress, then the long history of distance education and the more recent history of MOOCs can provide some useful guidance for predicting the future of a new instructor-guided technology. Since autograders can reliably evaluate human performance only in a few domains where the structure of human performance can be quantified and analyzed by a computer program, then you can predict that the new technology will be most helpful in science, technology, engineering, and math fields and less useful in the humanities and social sciences. If the new system you are examining doesn’t include substantial human coaching and advising, then it will probably serve well only a narrow slice of learners—those who have developed the self-regulation skills to navigate and persist through self-paced learning experiences. The students who tend to thrive in these kinds of environments are those who have already demonstrated academic proficiency, since most people develop self-regulated learning through an apprenticeship in the formal educational system. For those learners, self-paced learning can provide powerful, flexible learning experiences at low marginal (per-student) costs. But the risks that these kinds of technologies will accelerate rather than alleviate gaps in educational opportunity are quite high.
If an algorithm decides the sequence of learning activities, then your new specimen may belong in the long history of adaptive tutors and computer assisted instruction. Again, since these systems depend upon autograders, their utility is typically limited to a few fields—in the K–12 system, mostly math, early reading, language acquisition, and computer programming—where domain knowledge is amenable to computer assessment. Meta-analyses of adaptive tutors suggest that they can have positive effects in mathematics, and individual studies have shown benefits in other subjects. In particular, recent studies of Carnegie Learning: Cognitive Tutor and ASSISTments suggest that individual instruction with adaptive tutors might accelerate math learning; some studies suggest that it may even be possible to use adaptive tutors to address learning gaps between high- and low-achieving students. But because these gains are limited to a few subject areas, there is no realistic pathway to recreating whole-school curricula around these tools.4
If a community of peers creates the resources available to learners, the new technology that emerges from their efforts belongs to the peer-guided genre of learning at scale. In the world at large, these communities have dramatically reshaped how people participate in lifelong learning; in schools, the effects of these approaches have been more muted. The most powerful experiences in peer-guided learning at scale tend to be deep, collaborative, sustained, and interest driven. These characteristics, however, are at odds with the pedagogical approach of most schools, which usually require that learning experiences are pursued individually (not collectively), along a set of mandated curriculum guidelines (not determined by students’ interests), and for uniform timespans—the class period, the marking period, the semester (not sustained over time). The disjunction between the culture of informal online learning and the culture of formal educational systems means that schools struggle to integrate peer-guided, interest-driven technologies. Programs like Scratch or a Domain of One’s Own can gain a toehold in the periphery of educational systems, but the fit is often uneasy. The most powerful implementations tend to be in small pockets of innovation in a few classrooms rather than as part of schoolwide changes. These technologies can introduce new pedagogical ideas to schools, and they can spark dialogue about how best to prepare young people for a future of lifelong learning, but that is at best a starting point. Making open, networked, apprenticeship learning a central part of schooling requires rethinking all aspects of the ecology of schools, from curricula to assessment to schedules to teacher professional development and beyond.
Across all three learning-at-scale genres, predictions of disruption, transformation, and democratizing education through technology have fared poorly over the last decade, and indeed over the last century. Each of these genres has particular technologies that have proven useful in certain fields or for certain students, but new technologies do not disrupt existing educational systems. Rather, existing educational systems domesticate new technologies, and in most cases, they use such technologies in the service of the well-established goals and structures of schools. Two of the most reliable findings from the history of education technology are that educators use new technologies to extend existing practices and that new technologies tend to accrue most of their benefits to already-advantaged learners. With these two fact patterns in mind, and after using the four questions above, the analysis of a new learning technology can usually proceed on solid footing.
For all their differences, the three genres of learning at scale all interact with the same formal school system ecology. This intersection, between learning-at-scale technologies and formal education as it exists today, has three reoccurring features: complexity, inequality, and unevenness. These features help explain why learning-at-scale technologies do not simply improve learning for all students in all schools, and they are the source of the thorniest challenges in learning at scale.
Schools are complex systems, and many stakeholders in school systems—teachers, students, parents, administrators, and policymakers—are often quite committed to various aspects of the status quo. The schools that exist today are an assemblage of features designed to balance competing visions of the purpose of schooling: inspiring lifelong learning, helping learners pass through gatekeeping exams, preparing people for their lives as citizens. As a result of these varied purposes, schools are tasked with an almost inconceivable array of often competing functions: to teach people to read, to do math, to understand science, to stay healthy, to abstain from sex before marriage, to practice safe sex, to learn history, to love their country, to question their country, to play sports, to make art, to sing songs, to program computers, to work well with others, to become actualized individuals, and on and on. Each of these goals requires different kinds of curricula, learning environments, schedules, and instructional approaches. All schools choose to invest more resources in some of these goals than others. The utility of new technologies is uneven across these various goals; technologies have more traction in some domains than others. And on top of all of this complexity, our society allocates very different levels of resources to schools serving more and less affluent learners, and our schools all too often offer learning experiences of lesser quality to students from poverty-impacted neighborhoods or from ethno-racial minorities.
When emerging technologies are viewed against this background of social, cultural, political, and pedagogical complexity, it becomes clear why the gains and successes of learning at scale have a mishmash pattern—useful in this subject but not that one, with these learners but not those learners, in some contexts but not others. In most places, these complex forces conspire to limit the impact of emerging technologies. But there are certainly breakthroughs where thoughtful design, careful refinement, public demand, and other factors intersect in just the right way for massive numbers of learners to benefit from learning at scale. The online master of science program in computer science at Georgia Tech—the MOOC-based, asynchronous, online master’s that has become the largest computer science degree program in the country—appears to be effectively serving a population of working professional students who by all accounts wouldn’t or couldn’t pursue a master’s otherwise. The Scratch programming community has introduced millions of young people around the world—in schools and beyond—to computational creativity. Findings from large randomized control trials of ASSISTments suggest that it may be a lightweight online math homework helper that can lead to learning gains for all students, especially those who have previously fared poorly in math, with relatively modest investments of technology and time.
These exemplars are useful guides to future success stories in particular niches, but they are not harbingers of a transformation. One MOOC-based master’s program in computer science seems to be working, but great success seems far less likely from MOOC-based master’s programs in creative writing, nursing, teaching, or many other fields. The Scratch online community has made impressive inroads in schools. There are other online communities where young people develop new skills—the millions of young writers who engage in fan fiction creation stands out as a useful point of comparison—but I think it’s unlikely that many of these other online communities will find the same inroads in schools as Scratch has. Even if ASSISTments is a great homework helper for math, it is unlikely that it would work equally well as a homework helper for history, biology, or art class.
These exemplars and other efforts like them are limited by a common set of challenges to improving human well-being through learning at scale. In the second half of this book, I described these “as-yet intractable dilemmas,” which can also be framed as a set of questions that designers, policymakers, funders, and educators can use to forecast the challenges of improving learning with technology:
The curse of the familiar describes the challenges of introducing novel learning experiences into complex, conservative systems. Technologies that digitize existing school routines are easier to adopt, but they are less likely to meaningfully change schools; technologies that could meaningfully change and improve schools are hard for conservative systems to adopt. ASSISTments works in schools because it is designed to fit in typical schooling routines. But this alignment is also a limit; ASSISTments is helpful to math in schools to the extent that mathematics education is more about rote procedural fluency than more sophisticated mathematical reasoning. When a new learning technology does not reproduce typical schooling routines, educators often have trouble incorporating it into the curriculum. Scratch is designed to help learners and educators imagine computing as something much more creative than the procedure-heavy, syntax-heavy ways it is often taught in schools, but educators struggle to figure out how to make room for passion-driven, playful, time-consuming Scratch projects in the confines of a typical school day. The most promising approaches to these challenges have less to do with scaling technology and more to do with scaling communities of educators who can work together and learn together to do the hard work of reforming complex systems so that technologies can have greater impact.
The fact that large-scale learning technologies have an uneven impact across subjects can be traced to the trap of routine assessment. MOOCs, adaptive tutors, and other technologies that seek to assess and credential learners at scale depend upon autograders to computationally evaluate human performance. Autograders are unevenly useful across the curriculum. They are mostly useful in fields in which desired human performance is sufficiently routine for algorithms to reliably identify the features of high-quality and low-quality performance and to assign grades and scores accordingly—in math, quantitative parts of science, early language acquisition, and computer programming. Much of what we want students to learn, however, cannot be demonstrated through performances that adhere to these kinds of rigid structures. Indeed, in a world where humans are ever more rapidly transitioning routine tasks to robots and AI bots, the premium on creative problem solving and complex communication is growing. Our large-scale learning systems may be growing most rapidly in domains that will be least useful in the future, unless we can develop new ways to expand the subjects, disciplines, and skills that can be assessed meaningfully at scale.
The curse of the familiar and the trap of routine assessment help explain why learning-at-scale technology is difficult to integrate into the complexity of the current education system and why, when it is integrated, its impact is uneven across subjects and disciplines. The edtech Matthew effect helps explain why learning-at-scale technologies have uneven impact across people from different backgrounds. Across all three genres of learning at scale, when researchers evaluate how learners from different backgrounds access and use new technologies, it is common to find that the benefits of new technologies—even free technologies—accrue most rapidly to the already-advantaged. Early adopters of Scratch were likely to have parents who had some experience with computing. MOOC providers have shifted their focus to providing lower-cost master’s degrees because students who already have bachelor’s degrees are easier to educate than students looking for new pathways into higher education. This fact pattern is not an inevitable destiny, and designers can and should be exploring how to design technologies that are best at serving the students who have the least opportunity. But by the same token, educators should be wary of approaches that claim to rectify deep-seated structural inequalities through new technologies. Perhaps new technologies can play a role in creating more equitable ecologies of learning, but technology alone will not democratize education.
For education researchers, one of the most exciting features of large-scale learning technologies is that they can be changed and improved systematically; we can closely examine and digitally record how learners interact with digital platforms, and we can systematically test instructional variations within those platforms to see how competing approaches might benefit or harm learners. Yet some of the most promising approaches to this kind of research raise serious ethical questions: When should learners or schools consent to participating in educational experiments or assessments? Who should steward data from digital platforms, and what limits should be put on their use? Perhaps the most urgent question is, How might these systems of data collection and experimentation inculcate young people into accepting a culture of digital surveillance that could ultimately impinge on human autonomy even as it promises new freedoms and benefits? The toxic power of data and experimentation highlights that even if questions about edtech’s possibilities and potential are technical in nature, the questions of what we should do with technology are irreducibly political. In the long run, the best future for improving learning technologies through research will involve greater community involvement in addressing these tradeoffs.
I view these as-yet intractable dilemmas not as immutable barriers but as challenges for designers, developers, funders, researchers, and educators to rally around. What are viable design principles for digital equity? How could new assessment technologies provide more learners more useful feedback in more domains at scale? What are effective strategies for building communities of change agents devoted to improving teaching and learning through new technologies? How do we balance the possibilities of improving technology through continuous experimentation with the risks inherent in large-scale data collection and threats to the autonomy and dignity of learners? As I examine new announcements from edtech startups, research projects, and other new forays into learning at scale, I use these and similar questions as guides to identify what kinds of projects might be most likely to offer new designs or new insights that can address complexity, unevenness, and inequality and therefore could change the direction of learning at scale.
Improvements in education very rarely, perhaps never, come by way of dramatic transformations. They come through deep, long-term commitment to the plodding work of building more robust systems. Large-scale learning technologies absolutely can improve learning opportunities both in informal learning and in educational institutions, but lasting and meaningful change is unlikely to emerge through technologies alone, especially for learners with the least opportunity. Nearly all learning is situated in social communities—online networks, community centers, schools, and colleges—and learning improvements in those communities typically come from many interlocking adjustments; a new technology can be of value when schedules are adjusted to accommodate the technology, when goals and assessments are modified to align with what technologies are good at, when community leaders (teachers, moderators, coaches) develop new proficiencies with integrating technologies into their educational practice, and when the developers or peer contributors to a technology improve the product through iterative development cycles.
Consider Wikipedia, one of the most important learning resources in the world, with 18 billion views per month of more than 40 million article entries in 293 languages. It represents one of humanity’s most extraordinary achievements, a community-generated repository of global knowledge of almost incomprehensible scale: 27 billion words written, managed, and edited almost entirely by volunteers. When Wikipedia first found its way into schools, usually through students citing it or copying from it for homework, it was treated with deep suspicion; educators didn’t know exactly what it was, but they knew they hated it. But over time, reference librarians started peeking at Wikipedia to help address patron questions and sharing their insights with open-minded teachers, and slowly, the world’s encyclopedia has been accepted by many educators. The utility of Wikipedia has increased over time as the encyclopedia has grown, but also as communities of educators and learners have better understood how to use the resource.
Educators and experts periodically get together to improve specific elements of Wikipedia. For instance, Mike Caulfield, the director of online learning at Washington State University at Vancouver, recently led an initiative to expand Wikipedia’s entries on local and regional papers. Caulfield had observed a surge in viral fake news stories, circulated on Facebook and other social networks, that were crafted to look like stories from local or regional newspapers. Often, those fake news articles were attributed to publications that didn’t actually exist. Caulfield decided to strengthen Wikipedia’s entries on local newspapers so that citations claiming to be from newspapers that do not actually exist could be more easily and reliably vetted. Through a tiny, volunteer, citizen-educator-led effort, Wikipedia got slightly better, and the US information literacy infrastructure got ever so slightly stronger.5
Some communities work on the encyclopedia itself. Others work on curricula and pedagogical approaches to using the encyclopedia. Still others work on professional development for teachers and librarians about how to use the resource or how search engines like Google use Wikipedia as a framing device for many search resources. Through all these efforts, Wikipedia is becoming an increasingly valuable resource for learning and research inside schools and beyond. Whether this represents an educational breakthrough is up for debate—having this quantity of mostly well-edited information available in the pockets of most people in the networked world is an extraordinary achievement—but at the same time, it turns out that learning processes are so complex that having all of these facts in one place does not dramatically accelerate learning. It helps—it’s a valuable addition to our global learning ecology, and very few projects are likely to be as incredible a boon to learning globally as Wikipedia—but it is hard to make the case that young people in the United States or around the world are much smarter, wiser, more ethical, or better prepared for the world because of Wikipedia.
If you are hoping that new technologies will be able to radically accelerate human development, the conclusion that change happens incrementally is probably a disappointment. But if you think that global human development is a game of inches—a slow, complex, maddening, plodding process with two steps back for every three steps forward—then Wikipedia is about as good as it gets. New technologies get introduced into complex learning ecologies, and those complex learning ecologies require multiple changes at multiple levels to take advantage of new technologies. You can give people every fact in the world through Wikipedia, but people cannot make much use of those facts without improvements in instruction in literacy, math, research, self-regulated learning, and information literacy. As a result, changes in educational systems are necessarily incremental, but step change is what continuous, incremental change looks like from a distance.
New technologies can contribute to this ongoing march in two important ways. First, the technologies themselves can aid learning, be it in informal contexts or in formal settings. New technologies are rarely as transformative as we might hope (or as evangelists might promise), but to critique them for bringing only incremental change is not to diminish (all of) their value. If you believe, as I do, that educational improvement is a long, slow journey, it would be unwise to turn away from anything that might take us another step, and another step, and another along the path.
Second, the novelty of education technologies opens space for new conversations about the practice of teaching. The arrival of new learning technologies can be an invitation for communities of educators to look up from their critically important and engrossing day-to-day work and to imagine how a new tool might reinvigorate their practice. Techno-optimists will imagine new ways that learners can interact with content and peers, skeptics will point to the value in practices honed over generations, and in the conversations that emerge, we can find the particular places where specific technologies can provide some additional value and opportunities for learners that were not present before.
I find these kinds of conversations enormously enriching. In the K–12 system, new technologies for learning about computer programming—Scratch, Code.org, and others—have inspired schools and school systems to ask a whole range of important questions about who gets access to educational opportunities around computer programming, where computer science should fit into established curricula, how computer science teachers should be trained and licensed, how non-specialist elementary teachers can be supported in effectively introducing young children to computer programming, and on and on. One of the most generative things to come out of the surge of enthusiasm for MOOCs was a renewed interest in interrogating teaching and learning in higher education. At both Harvard and MIT, the arrival of MOOCs sparked or invigorated organizations such as Harvard’s Initiative on Learning and Teaching and MIT’s Office of Open Learning. At MIT, I recently helped teach a course called Designing the First Year Experience, in which MIT undergraduates participated in design efforts to reimagine the freshman year at MIT. The new possibilities of technologies opened broader conversations about learning across the institution. My lifelong commitment to understanding education technology is nourished not so much from the technologies themselves, but rather from the dialogue about pedagogy and curriculum that new technologies provoke.
If the energy and excitement generated by new technologies could be applied not just to technology, but to technology and system change combined, that would provide the best possible chance for the field of learning at scale to meaningfully improve how people learn in schools and beyond.