With some of the funding in place and some of the immediate financial pressures laid to rest, I was finally free to return to job one: education.
In September 2010, I’d been introduced to a man named Mark Goines, a prominent “angel investor” in Silicon Valley start-ups, and, more to the point as things turned out, a member of the Los Altos School Board. Los Altos is a wealthy town with one of the top school systems in California. It is also right next door to my own adopted home, Mountain View—if my house fell into the Los Altos School District, it would immediately be worth $100,000 more because of the schools. Mark and I decided to meet at a local coffee shop one afternoon.
We immediately hit it off. Mark was the type of person who made Silicon Valley what it is. He was super successful, super smart, and, most importantly, unassuming and down-to-earth. We talked a good bit about what the Khan Academy could do and the people it could reach. Half an hour into our conversation, Mark asked what I would do if I could totally reinvent the dynamics of a fifth-grade math class. Assuming this was a purely hypothetical question, I laid out my ideas.
Mark seemed to like what he had heard, but as we stood up after coffee, my assumption was that we’d had a pleasant chat and that was the end of it. Then he said that if I didn’t mind, he’d like to discuss my ideas with some other members of the school board.
I should mention in passing that at this juncture things were moving dizzyingly fast for the Khan Academy. It was already clear that Google and the Gates Foundation were going to support us in a big way, and it was making waves in the press. I was getting overwhelmed with meeting requests and the day-to-day of trying to get a real office up and running. I was also getting a bit worried that the whole reason for all of this attention—the videos—was taking a backseat to the nascent operations of the Khan Academy. I clearly needed help, and fast.
I convinced an old friend of mine from Louisiana and then from MIT, Shantanu Sinha, to formally sign on as president and chief operating officer. A brilliant guy who’d been shaming me in academic competitions since we were teenagers, Shantanu gave up a half-million-dollar-a-year, partner track position at McKinsey and Company to come aboard. I found it very reassuring to know that I wasn’t the only person crazy enough to give up a relatively safe and remunerative career in exchange for a long-shot chance at helping to rethink education on a global scale.
In early October, Shantanu and I met with Jeff Baier and Alyssa Gallagher, the superintendent and assistant superintendent for Los Altos schools. They listened to our presentation and realized we were proposing the kind of differentiated education—that is, teaching geared and nuanced to the needs of each individual student—that educators were always striving for but not quite knowing how to implement. They asked for some time to discuss our ideas with colleagues, principals, and teachers, and then suggested we meet again.
Five days later we got an email from Alyssa saying that they wanted to move forward and start a pilot program in four classrooms after Thanksgiving break—which happened to be a mere five weeks away. So Shantanu and I found ourselves in crunch mode—hiring first-rate designers and engineers, upgrading software, refining ideas. Let me emphasize why we were so passionate about this Los Altos opportunity. Khan Academy had been founded with the goal of reaching students outside of any formal setting, and we were already reaching a million students per month even before getting that first funding from Gates and Google. To a large degree, we were successful because we had the luxury of focusing 100 percent on end users rather than having to cater to school districts as some type of software vendor. Based on this, it could have been argued that the Los Altos project was a diversion or even a detour away from our student-focused mission.
But I, and eventually the rest of the team, always dreamt of being more than just a powerful online resource. We felt that we were at a point in history where education could be rethought altogether. We didn’t know all the answers—and still don’t—but the feeling was that we had to start experimenting in real settings so that we could at least be confident we were asking the right questions. We wanted to learn from real teachers and real students how our technology could be used or be made better. Los Altos was ideal because they were nonbureaucratic, open-minded, and located in the very heart of Silicon Valley. The fact that one of the best school districts in America felt that they could become even more effective by working with us was a huge sign of confidence that we took very seriously.
By the end of November 2010 the pilot program was up and running. Two fifth-grade classes and two seventh-grade classes were being taught math through the Khan Academy. No one, either teachers or students, had been compelled to participate in the program; we worked with the teachers who wanted to work with us. We’d held informational meetings with families and given them the chance to opt out; none did.
There were quite significant differences between the fifth-grade and seventh-grade classes. The fifth graders hadn’t been separated into “tracks” yet, and so were probably representative of Los Altos demographics—mainly English-speaking, with college-educated, affluent parents. By seventh grade, however, the students had been tracked, and our program was working with the “developmental” classes, the kids who had fallen behind. Some had learning disabilities; some struggled with English; few had college-educated parents. These students disproportionately came from the “other,” much poorer side of El Camino Real (the main avenue in Silicon Valley) that just happened to fall into the Los Altos School District.
But if the two groups had differences, they had similarities as well—mainly enthusiasm and curiosity. Now, as every teacher knows, there are things you can measure and things that you can’t. Energy level in a classroom is one of the things you can’t plot on a curve but is palpable and important nonetheless. And it was clear from the very start of our program that the energy level had been boosted. Kids were eager to start “Khan time” and some didn’t want to go to recess afterwards. They started exploring concepts on their own; they spontaneously began helping one another. In the seventh-grade classes as well as the fifth-grade ones, kids were starting to take control of their learning.
Part of the excitement was that for these students and teachers, the curriculum was developing before their very eyes. But they weren’t just watching it develop; they were actively participating in the process—not just accepting change but driving it. Ben Kamens and Jason Rosoff, our software designers who were now doing the heavy lifting on the engineering side, sat in on classes, seeing how kids were actually using and responding to the different features, tweaking this or that according to a teacher’s specifications. The feedback loop continually evolved. We started giving kids electronic achievement badges for advancing through concepts—a cost-free way to boost motivation and confidence. Kids came to realize that software was made by real people, and that education was not some monstrous, soulless weight imposed on them, but a living, breathing thing designed for their benefit and with their help. Forgive me for gushing, but there was magic going on in those classrooms, and the magic confirmed a belief I’d had ever since talking with my cousins about my earliest video lessons: that the best tools are built when there is open, respectful, two-way conversation between those who make the tools and those who use them.
But okay, it’s all well and good to talk about energy and magic and all those feel-good, California-style things; still, I was keenly aware that at the end of the day, the success or failure of the pilot program would be measured not in terms of these intangibles but by the hard-edged, flawed but inevitable, in-your-face criterion of performance on standardized tests. And I admit that as the day grew closer when our students would be taking their respective grade levels of the CSTs (California Standards Tests) I once again got pretty nervous.
But let me be clear about why I was nervous. It wasn’t that I had strong doubts that our kids were learning math. I was confident they were learning, and that, moreover, they were learning at a deeper and more durable level than most conventional classrooms afforded. My concern, rather, was with the congruence, or lack thereof, between what our kids were learning and what the tests were testing.
This is one of the paradoxes and potential dangers of standardized tests: They measure mastery of a particular curriculum, but not necessarily of the underlying topics and concepts on which the curriculum should be based. The curriculum, in turn, becomes shaped by the expectations of what will be tested. So there’s a kind of circular logic, an endless loop going on. Teach what will be tested; test what most likely had been taught. Topics and ideas and levels of understanding that go beyond the probable parameters of the test tend to be ignored; they aren’t worth the classroom time.
We were trying to enable learning in a different and, we believed, more organic way, a way aimed at conceptual understanding rather than test prep. Because we encouraged students to progress at their own pace, we had some very advanced fifth graders already working on algebra and even trigonometry. But this impressive advancement would go unrecognized on the CSTs, which only tested proficiency in the usual fifth-grade material. Further, with regard to the fifth-grade classes, we were up against some pretty tough comparisons, as 91 percent of students in conventional Los Altos classes were already testing as “proficient” or “advanced” for their grade level.
With regard to the seventh-grade classes, we had a somewhat different set of concerns. These students had been significantly underperforming their peers before participating in the pilot program; they badly needed remediation. Would our unconventional approach have provided it?
Test day came. We crossed our fingers and waited for results. When they came in, they were overwhelmingly positive.
Our fifth graders posted a stellar 96 percent at proficient or advanced grade level. I do have to say that a good bit of this outperformance was probably due to the amazing teachers in the pilot classes rather than just our resources. It did decisively prove to the district that despite the fact that our software was still at a nascent state and that we weren’t teaching to the test, the experiment was definitely not doing any harm. In light of the test results, coupled with the positive feedback from teachers, students, and parents, the board decided to use the Khan Academy as part of the math curriculum for all fifth- and sixth-grade math classes in the district for the following school year. In keeping with the Pinball Philosophy, we had done well at the game and so were being allowed to play again.
But the truly dramatic results were with the seventh-grade classes. Relative to a year prior, their average score on the grade level exam improved by 106 percent. Twice as many students were now at grade level. A handful of students jumped two categories, from “below basic” to “proficient.” A few even leapfrogged into the “advanced” category. As gratifying as these results were to us, it was equally pleasing to tap one more nail into the coffin of tracking. Our underserved, underperforming, and purportedly “slow” kids were now operating at the same—or higher—level as their more affluent peers.
I want to emphasize this last point. Remedial math classes are often viewed as something of an academic graveyard. Once students are deemed “slow,” they tend to fall farther and farther behind their peers. Now, all of a sudden, we were seeing that students who were put in the “slower” math classes could actually leapfrog ahead of their “non-slow” peers. Even better, the experience with both the fifth and seventh graders showed that there really was no reason to track students into separate classrooms to begin with. Now every student could work at his or her own pace; it was unpredictable who could eventually advance the most. It should be noted that this initial data came from a very small data set, a handful of classrooms, and was not designed as a truly controlled experiment. It did, however, point in a very promising direction.
By the summer of 2011, we began ramping up our team to manage a district-wide pilot with twelve hundred students in Los Altos. Many, many more teachers and schools were also eager to work with us. Given that we wanted to push our own learning and see how the Khan Academy could be applied in different use cases, we chose a handful of public, charter, and private schools in California that served very different types of students—seventy classrooms in total. Because all of the student and teacher tools we were using with our pilots were available for anyone to use, it became clear from our server data that there were also more than ten thousand teacher-led classrooms or cohorts, serving 350,000 students around the world, that were using us independently of any formal pilot program.
At the time of this writing, we are just beginning to get data from this larger wave of pilots, but the preliminary information seems even more exciting than what we saw from the first, limited pilot in Los Altos.
Let’s consider the Oakland Unity High School pilot, where 95 percent of the students are African American or Latino and 85 percent receive free or reduced lunch. First the subjective. In a recent blog post, David Castillo, the principal, and Peter McIntosh, a math teacher, wrote about how in previous years they “found that students failed to engage in the coursework and spent little to no time studying.” They went on to describe how “students were disengaged from their learning responsibilities and the derailing of their studying began as early as elementary school.” However, their descriptions of what is culturally happening in their pilot classrooms is exciting. They wrote:
We believe that our use of Khan Academy is resulting in a fundamental change in student character—with responsibility replacing apathy and effort replacing laziness. We believe that this character change is the primary reason behind the stunning results we are beginning to experience—at both the class level and in individual students.
And the data from the students’ tests scores is indeed exciting. Students are scoring 10 to 40 percent higher on average across a battery of exams covering different domains in algebra. The percentage of students showing reasonable proficiency in various topic areas is even more significant. For example, the percentage of students who now scored at least 80 percent on their recently administered “Systems of Equations” exam grew by a factor of four. It is perhaps too early to pick out a trend, but it looks like the relative improvement compared to prior years is only growing more dramatic as the class moves into more and more advanced topics.
We’re getting similar results from the other pilots. A group of sixth graders had entered the KIPP pilot from local Oakland public schools with a roughly third-grade-level mastery of math. Six months later, most of the class was operating at fifth- and sixth-grade levels. The teachers had never seen groups of students move ahead two and three grade levels in a matter of months. We are hoping to see much, much more data like this in the months to come.