Chapter 9

FIXING THE PEOPLE PROBLEM

A burglar, a spy, a fugitive, a delinquent, a hacker, and a piano teacher . . . and these are the good guys.

TAGLINE FROM SNEAKERS (1992 FILM)

Frank DiGiovanni is no cyber warrior. He is a real warrior. An ex–fighter pilot in the Air Force, he now works in the Navy as a civilian. The walls of his Pentagon office are adorned with the memorabilia collected over a lifetime defending the country: military decorations, photos of the teams he has been on, tactical knives, and innumerable challenge coins given out by military units in appreciation of his service. DiGiovanni has served as the director of Force Training in the Office of the Assistant Secretary of Defense for Readiness, as well as the assistant deputy chief of Naval Operations for Manpower, Personnel, Training and Education, and he is now the Deputy Director for Expeditionary Warfare. In all these roles, DiGiovanni has been charged with thinking about how the U.S. military trains to fight in cyberspace. He has come to some brutal conclusions about a field that many would agree is broken.

Breaking Through the Hype

The cybersecurity community is beset with hand-wringing about the field’s workforce crisis. To match every story about the cyber-expert gap, there is one about new programs to combat it. “The cybersecurity workforce is an industrial crisis!” declares Brian NeSmith, the CEO of a well-respected managed security service provider. And indeed, there certainly are a series of problems that we need to solve to get the workforce the nation requires. The first one, though, is to put an end to the hype. Breathless headlines declare workforce shortages of a million or more people. NeSmith quotes an often-used figure of 3.5 million unfilled positions by 2021, but the denominator is important. That figure is global, for the entire planet of 195 countries and 7.6 billion people.

There’s an unholy trinity of forces that have an interest in hyping the crisis. The first are companies that don’t like the high salaries that cybersecurity professionals demand. Attracting more people to the field, creating a slack in demand, and driving down costs would be of interest. Every time the issue of immigration reform comes up in Congress, the tech companies, in the hopes of expanding the H-1B visa program for technical workers, cite the cybersecurity workforce crisis as one of the main drivers.

The second player in this trinity is the cybersecurity industry itself, which does not necessarily want more workers in the field. They want their customers to buy more products and services from them, including managed security services that will outsource the work, automation of workflow, and AI that will replace the workers. The message is effective: You can’t hire the people you need. Spend your way out.

The last group pushing the workforce crisis are the security training programs springing up at every university and community college and competing against private training programs. All have a vested interest in making the problem seem out of hand, but what do the numbers really say?

The Cyber Census

When the good folks at NIST got tasked to help solve the cybersecurity workforce crisis, one of the first things they did was to get an accurate count of the existing workforce and the open jobs, down to a level of detail that would be useful. Then NIST created an online census of the cybersecurity workforce and job openings called Cyberseek, which brings real numbers to the workforce problem. What the numbers show is a far more nuanced story than what the tech companies, the toolmakers, and the training programs might have you believe.

NIST calculates the total cyber workforce in the United States at 768,096 people, and it identified 301,873 job openings in a one-year period. NIST then helpfully calculated that for every 2.5 people employed in cybersecurity, there is one additional opening. Nationally, across all career fields, there are 6.5 people for every one opening, so the cybersecurity talent market is most definitely tight. Where things get interesting is the data on certifications. NIST took the job postings and pulled out the certifications the postings requested. Then NIST compared them with the number of members of the workforce who have those certifications. What these numbers suggest is that the training programs endeavoring to get people into the field are solving the wrong problem.

Both of us regularly receive emails or are approached at conferences by people trying to break into the field. Many times, these individuals have no idea where to begin and are more interested in the policy aspects of cybersecurity. We generally give them the same advice. First, gain a technical grounding. Learn to work from the command line in Linux, learn the programming language Python, then take a penetration testing course. Increasingly, the people we talk to have taken several courses and obtained a series of entry-level certifications. They are applying to dozens of jobs. They are never getting a callback.

The reason for that appears to be that the demand is not at the entry level. The market wants more midcareer, experienced professionals. Looking at the data bears this out, as does talking to hiring managers. At any given company, the person in charge of cybersecurity first has to fight for money. When they get money, they have to fight for head count. Companies are often willing to spend big dollars on cybersecurity, but will fight to keep their teams as small as possible or even smaller than possible. If a director of security can staff up, the last thing they want to do is hire someone new to the field whom they will have to train for a year or two before getting any value out of them.

The data from Cyberseek bears this out: 167,776 people hold the CompTIA Security+ certification, an entry-level certification, against only 33,529 job openings that require it, for a ratio of 5 holders for every opening. If there are 6.5 workers for every opening in the overall economy, entry-level cyber isn’t in much higher demand than, say, retail workers.

Go further up the stack and the ratios start to shift dramatically in the other direction. Global Information Assurance Certification (GIAC) programs tend toward higher-level skills, with a historical connection to the NSA. There are 45,527 GIAC holders and 33,239 openings that request some of the GIAC certifications. It’s safe to assume that almost all of these GIAC holders already have jobs. For certified information systems security professionals (CISSP), certified information system auditors (CISA), and certified information security managers (CISM), there are more job openings than there are people with those certifications. Interestingly, for those who believe the skill shortage is only technical, auditors (CISA) and managers (CISM) are the two most in demand.

If the real shortage in the cybersecurity workforce is not at the entry level but at the level of experienced professionals, it is a much harder problem to solve. Pulling thousands of new people to the field, in many cases paying high tuition costs, will likely lead to a lot of people spending money on basic training and not being able to get jobs if new career pathways aren’t created.

The Cyber Workforce Market Failure

It is tempting to conclude that the cyber workforce problem will solve itself over time. After all, to move from an entry-level information-security analyst with a CompTIA Security+ certification to a CISO takes years. As more and more core societal functions go online and as companies either go out of business or make the leap to digital-first organizations, the scary headlines probably have some truth to them: the workforce gap is still growing and market forces alone are not fixing it. In almost all cases, we think that when markets fail, governments need to intervene. While sometimes government intervention means good, old-fashioned regulation, in this case there is probably no way to regulate our way out of the crisis. While cybersecurity is a twenty-first-century problem, we can look to two unlikely historical episodes for lessons: twentieth-century military challenges and the guilds of the Middle Ages.

When DiGiovanni got the task of figuring out how the military could train more (and more effective) cyber warriors, he first looked at how current military training programs in the field worked.

He came to a stark conclusion: the Department of Defense did not know who they were looking for and how to train a cyber warrior. “We were repurposing people from the comms community and the intel community rather than taking people who were born into the cyber community,” says DiGiovanni. “The military services were taking the easy way out by repurposing troops in similar areas rather than saying, what do I need to do this job, what are the attributes of the people who can do it, and what do I need to do to train them?”

As to identifying the people with the capability to become, in his words, efficacious cyber warriors, DiGiovanni quickly concluded that the basic military aptitude tests were missing the mark. “The tests we are giving people that simply measure intelligence are missing a big part of what it takes to be successful.”

DiGiovanni identified a series of core attributes he was looking for, based on interviews he conducted with those who were successful in the field at doing the work he needed to train his recruits to do, i.e., hard-core hackers. He talked to dozens of them. He now knew he was looking for people who had the ability to be self-taught, who learned better on their own than in classrooms. He was looking for people who became easily obsessed and would dig deep into a new subject (“It could be anything—coffee, brewing beer, cars—they just need to be passionately curious”), people who had the tenacity to work forever, to be the kind of people who will never give up on a problem until they have solved it.

They also had to be “prestige seeking.” They could be “breakers” in one sense of the term “hacker,” or “builders” in another sense of the same term, but they had to want to do things that others could not. They also had to be process oriented, creative but within constraints. Finally, they had to exhibit that most hackerlike trait, a willingness to constantly challenge the status quo, something the military’s culture does not exactly encourage.

DiGiovanni looked for candidates who fit these attributes within the military and the Pentagon’s civilian workforce. When he found them, he did not stick them in a standard classroom. He does not believe that most real-world skills are best learned by sitting in a chair, reading a book, listening to a lecture, and taking exams. For cybersecurity, he thinks these old-school learning models are hopeless, and yet it is how the U.S. military still trains most of its recruits to serve in cybersecurity missions. It’s also how most universities are approaching the problem. DiGiovanni, on the other hand, is a proponent of experiential and guided “autodidactic” learning. What that means is that he thinks cybersecurity is best learned by doing and is mostly about working on your own with some gentle guidance.

It may be that the best way to learn cybersecurity is to beat your head against a computer monitor for a decade in your parents’ basement, but the nation can’t wait that long. Thus, finding a way to identify students with the right innate abilities and then to move them swiftly along through a hands-on learning program may be a far better approach. With that insight, DiGiovanni created the Cyber Operations Academy Course. His first step was to hire a bunch of ex-NSA TAO guys.

TAO, for Tailored Access Operations, is the NSA’s legendary elite hacking team. They carry out focused operations against the world’s most hardened targets (other nation-states). They are the very definition of an advanced persistent threat and they hire their fair share of computer prodigies right out of high school and recruit heavily from the University of Maryland, MIT, and Stanford, with a compelling pitch: Come break into live computer systems legally, in the service of your country.

DiGiovanni came across a small company, Point3 Security, founded by an ex-TAO operator named Evan Dornbush and other ex-government colleagues, that was struggling to get its new endpoint system to the market. He gave them a simple pitch: Design me a course, and sit in the lab as a resource for students. Give them hints when they are stuck, but otherwise feel free to code away on your security application. And so they did, developing a series of progressively harder challenges for students to work on. There were no multiple-choice tests, no lectures.

When students got stuck, they could go to the instructors, who would mostly give them a simple piece of advice: “Google it.” They left coding manuals and trade publications around the lab as a resource as well, but the most valuable resource typically was other students in the class. The first year of the class was rough. Many of the students, drawn from across the DoD, military, and civilian workforces, dropped out and returned to their previous assignments. Some got the bug for cybersecurity and gained enough knowledge that they now have thriving careers in the military, federal government, and, in some cases, the private sector. What really made the program successful was that the Pentagon provided jobs for those who succeeded.

For their part, the Point3 guys did get their endpoint detection technology, Odile, finished and on the market. They’ve white-labeled it to a few vendors and are happy with what they built. Most of their time is now spent commercializing the course they built for the Pentagon. They created Escalate, a series of progressively more difficult challenges that students can take online. The first three are free. Beyond that, students, or the companies they work for, pay three thousand dollars a year for access to the program. As with the in-person course, mentors are available when students get stuck.

Dornbush, both as a trainer and a hiring manager, has seen the value of the approach that DiGiovanni pitched him. Point3 is rapidly expanding and he has a lot of slots to fill. “People come with all their certificates like CISSP,” says Dornbush. He gives them a challenge on Escalate and they sit there having no idea how to analyze the piece of malware he has dropped on them. They know what malware analysis is and probably aced all the questions on the CISSP about it, but they don’t know how to do it. “You want me to hire you to be a malware analyst; I gave you a piece of malware and you can’t analyze it?”

Point3 is, of course, not the only company with an online offering like this. Immersive Labs, a U.K. company, is taking a similar approach and has multiple customers at the large financial institutions. It’s a better and cheaper approach than in-person classes and can be used to assess the competence of current workforces, as well as provide them with additional training. This is probably not an approach that can, in a short period, turn out all of the highly skilled professionals the market is demanding. For that, students taking these type of courses evenings and weekends also need the opportunity to work on real-world problems by day.

Here is where the approach that the military took to new technical fields in the twentieth century is instructive. When Admiral Hyman Rickover created the nuclear Navy, he recognized that the program would never succeed unless he could guarantee that there would be no nuclear incidents as a result. Being a control freak, Rickover personally selected every officer who joined his program and decided that the Navy would train all its nuclear engineers in-house.

To this day, having an undergraduate degree or even a PhD in nuclear engineering would not help you get into the Navy’s nuclear training program. The only prerequisite for officers is to have taken calculus. Enlisted personnel are chosen based on how well they do on the Armed Services Vocational Aptitude Battery, which measures capabilities, not specialized knowledge. Similarly, taking private flying lessons is neither a requirement nor an advantage to becoming a military aviator.

For both aviation and the nuclear Navy, the military very quickly grew a large and capable workforce. That workforce, in turn, provided these trained individuals to the civilian workforce once they completed their service requirements. Go to any nuclear plant or talk to any civilian pilot and there is a good chance you will find someone who first received training and experience in the military. As DiGiovanni points out, when World War II began in Europe in 1939, the Army Air Corps could field only 1,500 aircraft. By 1944, the successor Army Air Forces had 80,000 aircraft and 2.5 million men and women under service. At the end of the war, as many of the pilots left service, the burgeoning civilian aviation market gobbled them up.

Today, U.S. Cyber Command has about eight thousand personnel. If there are over three hundred thousand job openings, U.S. military personnel leaving the services are far from being a main source of talent to fill that gap. Cyber Command doubtless needs to grow, but if we continue down a path where the private sector is responsible for its own security, there will never be enough skilled veterans for the civilian workforce. Instead, the approach of “recruit, train, deploy, and retrain” needs to be adopted by both the civilian government and the private sector.

Many in the Pentagon and civilian agencies will lament that as soon as they train up someone, the private sector will swoop in as fast as possible to hire them. Bobbie Stempfley, a former leader at the Department of Homeland Security’s Office of Cybersecurity and Communications, has often quipped that if the government is so bad at cybersecurity, then why did they constantly poach her staff? Yet instead of lamenting this churn, we should encourage it. There may be no better or more direct way for the government to support the private sector than by recruiting and training a cybersecurity workforce that private companies will eagerly hire. Creating this program in a civilian government agency like DHS would likely draw a different and possibly larger pool of candidates who are not inclined toward military service. After completing training, participants would have a multiyear obligation of service to pay back the costs, though private-sector employers should be allowed to buy them out of those obligations.

Cyber Guilds

Replicating a similar program in the private sector may be more difficult. Training new personnel from scratch is typically not something shareholders will find in their interest. To illustrate, let’s draw up a composite character from real people we know. She’s the CISO of a midmarket company. The company does $3 billion a year in business. It has eight thousand employees. She reports to the CIO, who oversees a one-hundred-person IT department. Three years ago she was the director of IT security and had no one reporting to her. Now, as CISO, she has two direct reports. Her board and CEO are all over cybersecurity and have upped her budget every year for the last three years. She can buy just about any tool she wants. What she can’t do is hire anyone.

Her CFO is tightly controlling head count. Employees are expensive. Employees introduce liability. Hiring a handful of junior people and training them is a nonstarter. If she gets approval for another person, she wants a “ninja” or a “wizard,” someone in the middle of their career who has experience across all the areas of cyber operations. A junior person would gain valuable experience working on the problems this company has, and could apply what they learn in an online program like Escalate at night each day (or by day each night, as they will probably get stuck on the graveyard shift). Here is where the guilds of yesteryear come in.

Learning by doing is, of course, not a new idea. Training programs in the form of hands-on apprenticeships were formalized in the Middle Ages, but they have largely fallen out of favor in the United States except in a few specialized trades (plumbing, carpenters, and electricians often still hire and train apprentices). When a presidentially appointed commission looked at the workforce problem at the end of the Obama administration, it concluded that the next President should create a national cybersecurity apprenticeship program with the goal of training fifty thousand new cybersecurity practitioners by 2020. The Trump administration has not moved this initiative forward.

Luckily, some of the private-sector leaders President Obama appointed to the task force have realized that the program need not be led by the government. Ajay Banga, the CEO of Mastercard, pushed leadership at Microsoft and Workday to join him in establishing the Cybersecurity Talent Initiative. Under the program, students who pursue a cybersecurity-related undergraduate or advanced degree will then do a two-year tour of duty in full-time cybersecurity roles at federal agencies such as the DoD, FBI, CIA, DHS, Treasury, and the Small Business Administration. They will then transition to positions with participating corporate sponsors. At the end of the program, the corporate sponsors will pay off outstanding student loan debt up to seventy-five thousand dollars. More than forty-five universities have committed to the program, from Harvard and MIT to a series of historically black colleges and universities. The founding partners committed to taking on a minimum of five program participants per year.

Of course, if only the founding companies join and only make their initial commitments to hire five a year, the initiative won’t do much to close the workforce gap. Alex Niejelow, senior vice president at Mastercard (and a former White House colleague of Rob Knake’s), who helped shape the initiative with Banga, hopes the program will grow rapidly. “We have three massive technology-enabled companies that are leading this initiative. We expect the rest of the market will follow,” says Niejelow. “Every company is facing this crisis.”

We applaud private sector efforts like this, but ultimately believe that government is likely to need to do two things. First, it needs to nudge the market away from requiring undergraduate degrees to get into the field and instead create pathways for professionals later in their careers. Today, many if not most of the best people we know working in even the most technical areas of cybersecurity do not have undergraduate degrees in computer science. If the national security of the United States requires college freshmen to make good decisions about their career prospects, we are likely doomed. Thus, we think an intensive boot-camp program like what DiGiovanni pioneered is likely a better pathway into the field than emphasizing formal education. Second, government may need to subsidize private companies to provide apprenticeships, as the federal government alone is not likely to be able to provide enough meaningful initial positions.

Of course, risk is inherent to the approach that we have outlined. We might train hundreds of thousands of people in cybersecurity only to have them replaced by robots. Artificial intelligence is making gains in automating some of the entry-level tasks that Tier 1 analysts do, like being “the eyes on the glass” in the Security Operations Center (SOC). Indeed, it’s possible that in a decade or two, the cybersecurity workforce may actually shrink. Given that Apple, Google, and Amazon have invested billions of dollars in digital home assistants that still can’t compose a grocery list, we think the point at which trained cybersecurity professionals are going to be looking for jobs is pretty far off.