Times and conditions change so rapidly that we must keep our aim constantly focused on the future.
—WALT DISNEY
I n the summer of 2016, I rarely left my house without seeing a group of people walking around randomly in packs, looking down at their phones. Often they’d stop, back up, and point excitedly at nothing in particular. Then they would exchange highfives and strike up conversations with one another even though they were often complete strangers. This was my informal introduction to the mobile-based augmented reality game, Pokémon Go . It was amazing to me that all these kids had gotten off the couches and embraced the outdoors, all in an effort to find and catch little digital monsters called Pokémon.
Being in the tech industry, I knew of augmented reality (AR), but what I didn’t know was just how powerful a tool it could be for motivating people to go into the real world and do stuff . I witnessed, with my own eyes, teenagers voluntarily going into libraries and museums! Of course, I live in Silicon Valley, where kids do all sorts of strange things. I wondered: How many people are playing this elsewhere? Well, a lot, as it turns out. The Pokémon Go app set the record for most downloads ever in our App Store—and it did this in its first week ! Within two weeks of its launch it had amassed over twenty million active users (more than Twitter), becoming the largest mobile game in U.S. history. Pokémon Go ’s historic explosion of players first showed me the true potential of AR, not for games, but as a motivational tool. I knew the implications of this for education were enormous and I believed then, as I do now, that augmented reality will very soon play a significant role in what that future looks like.
AUGMENTED REALITY
Augmented reality overlays computer-generated content right smack onto our real world. In Pokémon Go , for example, small representations of “Pokémon” appear on players’ phone screens as they move through the physical world, based on GPS signals, and players tap the creature on the screen to capture it. The AR part comes when you tap a button that turns your camera on and allows you to see the Pokémon superimposed onto the real world. Whatever you’re looking at through your phone’s camera (your friends, church steps, a library shelf) will appear to include that Pokémon.
Virtual reality, augmented reality, and now mixed reality (MR) are all part of a broader field called immersive technology, also known as immersion . Since most people are more familiar with the idea of virtual reality than they are augmented reality, I have found that the simplest way to explain the difference between the two is this: VR uses headsets to place us inside of digital worlds, whereas AR brings the digital world to our physical world, and MR is a combination of both.
When it comes to VR, we have already seen some of the success it can have just based on the Beijing experiment results discussed in chapter thirteen . VR’s 360° computer-generated environments are fully immersive 3D worlds that, hopefully, will one day enable teachers and students to go on virtual fieldtrips anywhere in the world, in the past, present, or future. Rather than the teacher bringing a T. rex into our world, as in AR, kids would be transported, via goggles, glasses, or headsets, to the T. rex ’s world, where both it and its natural environment could then be studied. The long-term potential of VR is strong. I like the idea of giving students their own personal time machine, where just by wearing a headset they can go back eighty million years to the Cretaceous Period to walk among full-scale dinosaurs in photographic-quality 3D. Leading the way in this field are Facebook, with its Oculus Rift; Microsoft, with its HoloLens; and Google, with both Google Cardboard and Daydream headsets. Apple has been doing research and development in VR for years, but they are much more excited about AR. 80
While the future of VR may indeed be bright, I’m personally and firmly in the AR camp for now, for two primary reasons. First, I’m not a big fan of kids completely disappearing into virtual 3D worlds. While it’s actually very cool to experience VR in short bursts, we all know that these things (especially games, which always come first) will be highly addictive, and we may end up never seeing our kids again! I’m only slightly joking, of course. The second, more practical reason is because of what’s quickly becoming possible for students, not tomorrow, but today. Rewiring education is not about predicting the potential of students and technology in some hypothetical future; it’s about unlocking all of this potential today.
One of the most impressive schools I have visited, in terms of its ability to properly utilize technologies like augmented reality, is the Varmond School in Morelia, Mexico. Run by the school’s founders, Noel and Noemi Trainor, Varmond is a PK–8 school built from the ground up to integrate technology throughout every aspect of its curriculum. Set within an adaptive, challenge-based learning structure, Varmond uses augmented reality, and other cutting-edge technology, in most of its classes. It has a 3D maker lab, interactive books, and mobile devices for all teachers and students. And every one of the learning spaces highlighted in chapter six is seamlessly integrated throughout the school. In fact, of the thousands of schools that I’ve visited in my career, I have never seen one that has successfully integrated transformative technologies, plus just about every other best practice, as well as Varmond. In many ways, the school of the future is already here and has been one of the best kept secrets in education. 81
Over the past decade, several organizations have made good use of AR at every level of education. One of the most pioneering of these has been the nonprofit television network PBS. As far back as 2010, PBS Kids launched an online AR game called Dinosaur Train Hatching Party , where kids had to traverse a combination of real and virtual worlds to help hatch a dinosaur egg. A year later, PBS Kids launched one of the first multiplayer, augmented reality 3D game apps, called Fetch! Lunch Rush , in which kids had to try to keep up with the lunch orders of a cartoon dog’s movie crew. According to a press release at the time, the game “opens a new world of learning by teaching kids ages six to eight math skills, like addition and subtraction, while blending the virtual and real world into a truly engaging experience.” The game fit perfectly into PBS Kids’ goal for the technology. “Our goal is to use media to nurture kids’ natural curiosity,” said a PBS Kids senior vice president at the time, “and inspire them to explore the world around them.” 82
Another early innovator in the AR educational space was NASA, which in 2012 released its own AR app, Spacecraft 3D , where students could use their mobile phones to view 3D renderings of several NASA spacecraft right in front of their faces. These are just a few of the pioneering efforts in a field that is very much emerging. Since those early days, AR has quietly grown into a powerful, if still somewhat unknown, technology. More recently, however, things are really picking up.
In 2017, almost exactly one year after the initial Pokémon Go craze hit, Apple announced the launch of a new iOS-based AR developers kit, called the ARKit, which allows third-party iPhone and iPad app developers to easily incorporate augmented reality inside their apps. 83 This means not just Pokémon will be able to “appear” in your house or classroom anymore, but anything can. The options are virtually unlimited and include images, data, charts, graphs—you name it. Now we can position virtually any of these objects to look like they are actually being placed in real-world spaces. I don’t mean just hovering over the space either, I mean in spaces. Because of an embedded technology called SLAM (simultaneous localization and mapping) and depth-sensing cameras, putting virtual things into real-world spots comes down to inches.
One of the best things about AR, from an educational standpoint, is that it can superimpose much more than just holographic images. It can also project maps, graphs, videos, and text, such as fun facts, definitions, statistics, and online comments. On a mobile device, these holographic projections can be triggered by using the device to view or scan an image or QR code, or upon arrival to a certain location, through geotagging . There are many other ways of seeing these holograms, too—through an AR headset, a digital camera, or a computer monitor. In each case, the device imprints the images on the lenses, in digital layers that can be added or removed by the user on demand. Through AR, all types of static content can be brought to life, from the periodic table, to biographies, to space exploration.
So, what can this do for students? Let’s take a look at one scenario, where everything discussed is possible now, or will be within the next year or two. Remember the boy we met earlier who loved space and used Siri to instantly answer his questions? What’s he supposed to look at and study in the daytime when you can’t see most stars? Well, now he has a solution. All he would have to do is turn off the lights in his living room, launch a solar system AR app on his iPhone or iPad, and watch as the solar system emerges over his own furniture. As he then walks through his living room, comparing galaxy formations being projected in 3D by the app, he can also zoom in and out of these galaxies, in order to take a closer look at a specific star or planet, and use his finger to actually manipulate them, turn them around, and see them from various angles. The app also projects text, on demand, that highlights certain points of interest for him.
But wait—what if he wants to know even more about Andromeda than what’s being projected in front of him? Well, that’s where things really get interesting. Because the ARKit and the apps that it enables are a part of his iPhone or iPad, as opposed to a large, uncomfortable headset he has to wear, this means that everything that his phone can do could also interact with his solar system app. So if he has a question, all he would have to do is ask Siri, and he’ll get his answer in seconds. And just in case all of that is not exciting enough for education enthusiasts to think about, remember this is all done via mobile, meaning learners can do all of these same things wherever they are. They are unbound by both time (apps are available 24/7) and location (mobile allowing us to be quite, well, mobile ).
That should shed some light on why I’m so excited about the potential of AR to become one of the most transformative educational technologies to ever exist. Take just a moment to imagine the possibilities of combining AR with things like 3D printing, artificial intelligence, adaptive learning, wearables, the Internet of Things, robotics, GPS, big data, biometrics, social media, on-demand, livestreaming, crowdsourcing, cocreation, and the sharing economy. I get excited just thinking about how any one of these can work with AR to improve learning, and all of this amazing technology is already here. That’s right, everything described about AR so far is either possible now or will be very soon. The technology itself is ready; someone just needs to get on the ball and make that cool solar system app!
Looking a bit further ahead at the future of AR, the prospects of possibility become even more intriguing. Over the course of the next year or two, teachers (and parents) will be able to download mobile apps containing interactive AR content on virtually any topic, as apps that support user-generated content allow for even the most tech-challenged of us to add and share valuable content with learning communities around the world. If nothing that already exists fits a teacher or parent’s need, then they will be able to create, and easily test, their own dynamic AR content and adjust its complexity to fit the level of individual students. Developments, like the ARKit, are turning augmented reality into a platform, and where platforms exist, ecosystems are never far behind.
THE MAGIC OF HOLOGRAMS
While I’m not a fan of making long-term predictions about the rise and popularity of technologies, I’ll play along for just a moment and share with you just a bit of what industry experts foresee. And from what I’m hearing, we’re only about five years away from these becoming more of a reality too: holograms . If this type of AR comes to fruition on a large-enough scale, we won’t even need to look at your phone’s screen to see holographic images; they would just appear in full 3D right there in front of us.
As graphics continue to improve, holograms will begin producing images in photographic quality and, one day, be able to render it in a 4D environment (another emerging technology), where other senses, like smell, sound, taste, and touch, will also play a role. Instead of reading about dinosaurs, imagine learning about them through a full-scale holographic projection of a T. rex that can be seen, felt, heard, and even smelled, as if he were right there in the school with you. At museums, statues will be able to come to life, relay their stories, and use AR and AI to conduct open question-and-answer sessions. Learners will be able to physically interact with AR projections, moving them around just as they would material things.
One of my favorite promotional videos in recent years has been one created by the augmented reality start-up, Magic Leap. Founded in 2010 in Plantation, Florida, by Rony Abovitz, Magic Leap seemingly came out of nowhere with what appeared to be some of the most advanced mixed-reality technology anyone had ever seen. It wasn’t long before word of mouth started to spread about what they were up to and all sorts of wild rumors circulated. Abovitz seemed to relish Magic Leap playing the role of “The World’s Most Secretive Startup,” as a 2016 Wired magazine article had dubbed them. 84 While very few people had actually seen Magic Leap’s technology themselves, the buzz around it grew so big that it was able to raise over $1 billion of funding, including from some of the biggest names in venture capital, such as Google and Alibaba. In 2016 Forbes valued Magic Leap at $4.5 billion. Not bad for a start-up that has yet to release a single product!
The thing that really took Magic Leap to the next level in everyone eyes, though, was a simple, two-minute demo video that was set as their website’s home page and ran in a continuous loop for anyone who visited. The video loop opened by showing a high school gymnasium filled with students, sitting on bleachers, staring out at the empty gym floor. Then, suddenly, a life-size whale comes crashing up through the gym floor, jumping high into the air, right in front of the students who stare up at it in disbelief, clapping and screaming, “Whoa! Ooh! Ahh!” Crystal-clear water appears to splash all over the gym, soaking the entire gym floor as the whale crashes back down into it and completely disappears. Seconds later all of that water vanishes too.
“Whoa!” is right! This amazing video purportedly showed that the future of AR and holograms was already here; it’s just that everyone else in the world didn’t realize it yet! Unfortunately, while some of the other cool but more realistic video clips in the loop were real, the holographic whale splashing the kids was simply a promotional gimmick. But I take the time to mention this in detail because I believe strongly that the life-sized, virtual whale in that gym, splashing those kids with virtual water, really is the future of learning . I foresee that as being exactly how students will be able to learn at some point in the future, through AR. As for the video itself, I believe magic can still be worth watching, even if we do know how it’s done.
Anytime I think about holograms today, I can’t help but think back to nearly thirty years ago, when my son, Kris, was in the eighth grade. He had a science project to do, and the topic could be anything of interest to him. Kris picked holograms, which were not very well known back then. But he thought they were the coolest thing ever, so I took him to the MIT Media Lab, and Kris got to meet and talk to some of the hologram researchers. He was so excited! “Holograms are the future, Dad,” he said on the way back. I barely knew what they were, so I just chuckled and said, “Okay.” This is just another example of how digital natives (even twelve-year-olds) are always one step ahead of us when it comes to technology!