We are what we do. Excellence, then, is not an act, but a habit.
—Aristotle
An April 2021 headline in the Wall Street Journal read: “NATO Wargame Examines Cyber Risk to Financial System,” followed with, “Financial industry helped plan scenarios in which widespread disruption would hit banks and other firms.”1
The North Atlantic Treaty Organization (NATO), with more than 2,000 participants from 30 countries, ran its annual Locked Shields wargame exercise on April 13–16, 2021. For the first time ever, the scenario explored how widespread attacks on a fictional nation's infrastructure might strike at activities critical to keeping the global financial system functioning.
From Mastercard to NatWest Group PLC to the Swiss Computer Emergency Readiness Team, numerous experts planned scenarios to help test emergency response plans and examine how ready financial teams were for unplanned disruptions.
But this four-day event was more than just a tabletop exercise, where executives typically sit around a table and discuss how they will handle various emergency situations. NATO called this simulation a “live-fire” exercise, which involved actual attacks against systems set up with cyber teams defending against the attacks.
Although this exercise was the largest such global exercise of its kind to date, an earlier series of “Quantum Dawn” exercises (the latest being Quantum Dawn V)2 tested similar controls and financial organizations on a smaller scale.
Organized by SIFMA, which is the voice of the United States's securities industry,3 Quantum Dawn V “enabled key public and private bodies around the globe to practice coordination and exercise incident response protocols, both internally and externally, to maintain smooth functioning of the financial markets when faced with a series of sector-wide global cyberattacks. The exercise helped identify the roles and responsibilities of key participants in managing global crises with cross-border impacts. The exercise scenario emphasized cross-jurisdiction communication and coordination between member firms and regulatory agencies in North America, Europe, and Asia… .”
In the exercise press release after the 2019 Quantum Dawn V global exercise, the following lessons learned (recommendations) were noted for the financial industry:4
Why spend time, money, and other resources on your organization's cyber exercises. What is truly at stake?
To gain a glimpse at the scope of the challenges faced by public and private sector organizations, the January 14, 2021 “DHS Strategic Action Plan to Counter the Threat Posed by the People's Republic of China”5 report reveals quite a bit.
Following are a few small excerpts, but the entire report is worth reading:
In an increasingly digital and interoperable world, we face expanding threats to our cyber networks and critical infrastructure in scope, scale, and frequency. The Department's Components, led by the Cybersecurity and Infrastructure Security Agency (CISA), are acutely aware of these risks, particularly those emanating from the PRC. While CISA plays a central and cross-cutting role across our Nation's critical infrastructure, the Transportation Security Administration (TSA) and U.S. Coast Guard (USCG) also play a key role in bolstering resilience to cyber and emerging technology vulnerabilities in the transportation sector….
CISA maintains a close and collaborative partnership with industry tailored to bolstering cyber-mitigation practices against adversaries, from nation- and non-state actors alike. This unique private sector partnership approach enables CISA to disseminate cyber-threat information, host training, and personnel exchanges, and circulate threat bulletins and alerts. Recent products have included spotlighting PRC-affiliated cyber threat actors targeting U.S. government agencies, PRC malware variants used to attack and maintain a presence on U.S. victim networks, as well as targeting and attempted network compromise of healthcare, pharmaceutical, and research sectors working on the COVID-19 response and vaccines… .
Increase Resilience of the Homeland to Nation-State Threats: Critical Infrastructure Assessment: DHS PLCY will bring together FEMA, CISA, and other key DHS Components to understand the potential impacts associated with PRC threats to critical infrastructure and assess any gaps in current DHS National Preparedness and Planning activities. Expand Homeland Partner (FSLTT and Private Sector) Threat Trainings and Exercises CISA and FEMA will augment cyber and physical trainings and exercises with FSLTT.
CISA and FEMA will augment cyber and physical trainings and exercises with FSLTT and industry partners to strengthen the resilience of critical infrastructure targeted by the PRC. This includes incorporating PRC Tactics, Techniques, and Procedures (TTPs) and other “real-world” nation-state scenarios into large-scale cyber exercises, including Cyber Storm VII and Cyber Storm VIII… .
The Spring 2021 Colonial Pipeline and JBS Foods events showed the world what a failure to protect critical infrastructure looks like. Cyber exercises strengthen cyber preparedness in the public and private sectors by offering solutions that mitigate these very real threats.
Just as military, police, and fire professionals have trained over the centuries in various scenarios, a new push to test cyber defenses began two or so decades ago at the U.S. Department of Homeland Security. Shortly after the formation of the agency, accelerated efforts began to look at critical infrastructure protection (CIP) in many industries, and the significant cross-cutting nature of cybersecurity was recognized with the addition of Cyber Storm.
If you visit the Department of Homeland Security's website at www.dhs.gov/cyber-storm, you will find the following description of Cyber Storm exercises:
Cyber Storm, the Department of Homeland Security's (DHS) biennial exercise series, provides the framework for the most extensive government-sponsored cybersecurity exercise of its kind. Congress mandated the Cyber Storm exercise series to strengthen cyber preparedness in the public and private sectors. Securing cyber space is Cybersecurity and Infrastructure Security Agency's (CISA) Cybersecurity Division's top priority.
Cyber Storm participants perform the following activities:
- Examine organizations' capability to prepare for, protect from, and respond to cyber attacks' potential effects;
- Exercise strategic decision making and interagency coordination of incident response(s) in accordance with national level policy and procedures;
- Validate information sharing relationships and communications paths for collecting and disseminating cyber incident situational awareness, response and recovery information; and
- Examine means and processes through which to share sensitive information across boundaries and sectors without compromising proprietary or national security interests.
Each Cyber Storm builds on lessons learned from previous real-world incidents, ensuring that participants face more sophisticated and challenging exercises every two years.
The portal also offers the reports from each of the previous Cyber Storm exercises, including an executive description, goals and objectives, participants, scenario, and an exercise final report.
In a 2018 blog post by Jeanette Manfra, the former Assistant Director for Cybersecurity for the department's Cybersecurity and Infrastructure Security Agency (CISA), the importance of cyber exercises is highlighted:
Cyber threats to government networks and other critical infrastructure are one of our Nation's most pressing security challenges. Consequences from attacks threaten the safety and security of the homeland, our economic competitiveness, and our way of life. With the majority of critical infrastructure owned and operated by the private sector, securing cyberspace is only possible through close collaboration, what we described as a “Collective Defense” model of shared responsibility.
Exercises are critical to testing this coordination, and more importantly, to building and maintaining strong relationships among the cyber incident response community. Carried out regularly, these exercises allow us to achieve solutions to some of the biggest challenges facing the homeland as well as raise the overall profile of cyber events and cyberattacks. …6
We recommend that anyone involved in cyber exercise planning read through the Cyber Storm final reports that go back more than a decade, including the latest report from Cyber Storm 2020.7
While Dan was the CISO in Michigan, the state was invited to be a player in the first Cyber Storm exercise, held in 2006. The technology and security teams from across the Michigan Department of Information Technology (MDIT) worked with the Michigan State Police Emergency Management Division, the Multi-State Information Sharing & Analysis Center (MS-ISAC), the U.S. DHS, and several other states and federal agencies to prepare for many months – starting in 2005.
MDIT had its own internal incident response plans, strategic and tactical cybersecurity plans, best practice guidance from NIST and others, and the team thought they were ready to go.
But they were very wrong. The scenarios were deadly, and the team was not prepared.
The first two days of the exercise were brutal, with events that were (in hindsight) over the top: Explosions, such as terrorists blowing up one of their data centers and bombs going off all over town, left the team reeling. Almost all of their capabilities were quickly disarmed or taken away via hacked computers – creating a hopeless feeling.
By Day 4, the team was very tired and just looking forward to the end of Cyber Storm I and getting back to their day jobs. But there was one final task. They needed to get a Bull mainframe back online to process employee payroll and perform other essential tasks.
Why? In the scenario, one of the Bull mainframes was blown up, and the backup was rendered useless by a series of cyberattacks.
But how could they get the mainframe online? After temporary paralysis and puzzled looks in the team room, someone discovered the (simulated) phone number for Bull headquarters in France in the reference materials available.
Dan phoned this number, and someone with a thick French accent answered the phone. The conversation went something like this:
Dan put his hand over the phone, as he spoke to the exercise team in the room. “They want $45 million.”
The room exploded with angry shouts: “What?!”
“Who do they think they are! That's extortion.”
After some back and forth over the next 10 minutes, they negotiated the price down to $23 million, and the cyber exercise ended shortly thereafter.
But the next day, Dan's team held a “hot wash,” where they went over the Cyber Storm I exercise with planners, several DHS monitors, and others who participated. They covered the good, the bad, and the ugly from the week, including lessons learned.
One staff member raised his hand and said, “The Bull mainframe scenario. That would never happen in real life. That was extortion. We were almost being held for ransom. No hackers would ever do that during a cyberattack!”
Someone else blurted out in a mocking tone, “Yeah. Ransomware!” as everyone laughed and agreed that this was ridiculous.
Little did anyone know that within a decade, ransomware (in different forms) would become a top cyberthreat. By 2020, during the COVID-19 pandemic, ransomware grew exponentially to become the top challenge for most technology and security teams in global enterprises.
Later, statewide cyber exercises in Michigan (and in many other states and countries) brought in the private sector to test hospitals and other organizations during a pandemic.
A 2008 article in CSO magazine discusses some of the takeaways from participation in Cyber Storm II.8 Later cyber exercises blended in street protests and other scenarios, with utilities and other private sector organizations also testing their cyber response plans.
By 2014, testing of incident response plans brought in other components such as the Michigan Cyber Range and the Michigan Cyber Civilian Corps to test the Michigan Cyber Disruption Response Strategy.
As mentioned in Chapter 2, NIST offers generous guidance in creating organization incident response plans for cyber emergency response situations. Another helpful resource is Mitre's Cyber Exercise Playbook.9 This playbook covers tabletop exercises, hybrid exercises (including scripted injects [unexpected twists that are thrown into exercises] and real probes/scans), as well as full live exercises (including real and scripted events).
Helpful examples include:
CISA also offers extensive support via their National Cybersecurity and Communications Integration Center (NCCIC), which develops and supports integrated cyber incident response plans and guidance and cyber-focused exercises for governmental and critical infrastructure partners.10 NCCIC's National Cybersecurity Exercises and Training conducts a full spectrum of exercises in cooperation with the public and private sector and international partners, particularly those who support U.S. critical infrastructure.
Additionally, the Center for Internet Security offers six tabletop exercises to help prepare your cybersecurity team for inevitable cyber incidents.11
If you do a Google search for “cyber range,” you will get over 400 million results, but a decade ago the “cyber range” trend was just beginning.
A cyber range is a controlled virtual environment where students can practice using their cyber skills without real-world negative consequences. From high school students to expert “white hats” with decades of experience, hackers can hone their skills and practice attacking and defending different systems in much the same way that a shooting range allows police to practice using firearms.
There are many state government cyber ranges, like the Virginia12 and Michigan13 cyber ranges. There are numerous private sector cyber ranges, like Palo Alto14 and IBM15 cyber ranges. In addition, there are many university cyber ranges, such as the U.S. Cyber Range hosted at Virginia Tech.16
But this was not so a decade ago. In Chapter 2, we described the Michigan Cyber Initiative. The Michigan Cyber Range was launched in 2012 as part of that wider effort after a meeting with Howard Schmidt, who was the Cybersecurity Coordinator and Special Assistant to the President.17
With support from the public and private sectors, the Michigan cyber team was encouraged by our meetings with representatives from the National Institute of Standards & Technology (NIST), the Department of Homeland Security (DHS), the Department of Energy, and others to create a first of its kind unclassified cyber range to assist not only in training and cyber exercises, but also in enhancing, cyber strategies, test tactics for teams working together in cyberdefense, and much more.
The concept was, and still is, in many cyber ranges, to test not only traditional computer systems, but also a wide range of Internet of Things (IoT) devices and virtually anything that connects to the Internet – prior to going live in the real world. Teams of technology leaders from within government, the private sector, and academia met with companies from around the state and country to encourage support of these cybersecurity efforts, and the response was very positive.
Cyber ranges are also available for virtual training and onsite to simulate various scenarios. The Michigan Cyber Range created cities, called Alphaville and Griffinville, to assist in the cybersecurity training:
Alphaville is an unclassified virtual training environment that resides in a high-capacity network. It is accessible from anywhere in the world and exists within a private cloud operated by the Michigan Cyber Range. Each location within Alphaville features different operating systems, security priorities, and challenges.
These locations include a City Hall, Public Library, Public School, Alphaville Power and Electric Company, and Zenda, a small engineering and manufacturing business.
This environment features:
- SCADA
- Security Tools and Appliances
- Email, File Sharing
- Permission Management and Access Controls
- DNS and BGP
- Various vulnerable databases and websites
- Alphaville is configured and misconfigured to specifications
- Ability to be customized
Camp Grayling is a brick and mortar town that the Michigan National Guard uses to conduct training exercises. Griffinville is a 3D virtualized representation of Camp Grayling built with Unity. Our plan is to incorporate physical PLCs such as water pumps, door locks, and electrical systems into Camp Grayling and tie them back to the virtual infrastructure back at Merit.18
In the military, continuity of operation plans is built into their ethos, training, and execution. Some of the best business continuity plans (BCPs) have come from those who have either retired or left the military and joined the public sector.
There is a military saying, “You don't exchange business cards during a crisis.” One should plan ahead of time so that people know what needs to be done, and the different players who should take ownership of the different roles.
It takes time to do this organically within the organization and to identify all the players needed, from law enforcement agencies to third-party partners. Think of the most likely and relevant scenarios (e.g., what could potentially happen) and then build a plan of action around that. This group should include legal, finance, business leadership, the technology internal team, technology vendor partners in your supply chain, and potentially even key clients.
Businesses seldom take time to benchmark against each other in the same industry. However, in particular, those who are very successful have proactively reached out to colleagues who are in similar businesses. Although some might have the view that they should not talk to the competition, there is huge merit in opening up a conversation, exchanging experiences, and benchmarking their BCPs against each other.
Yuval Illuz is the group CISO and COO of Trust, Data & Resilience at Standard Chartered Bank, a global bank headquartered in the UK with the majority of its global business leadership based in Singapore. He currently leads more than 2,000 employees globally across cyber security, business continuity, and operational resilience for the group. Within a span of 18 months, his team has had to manage the global pandemic and strengthen its capabilities in the face of growing cyberattacks and natural disasters like floods that could impact the bank, just to name a few. His other responsibilities include data privacy, analytics, data monetization, AI and machine learning, training and awareness, and third-party security.
Yuval walked through a key lesson of the pandemic when it comes to business continuity. “The pandemic teaches us that the path to an overall stronger cybersecurity is agility. This means having a more flexible cybersecurity architecture, helping our technology teams easily deploy the appropriate ICS (information and cyber security) controls as a significant portion of our workforce went remote over the past 18 months.
“It also means investing in operational resilience as we grow the trust and loyalty with our clients. Operational resilience refers to the ability of organizations and the financial sector as a whole to prevent, adapt, respond to, recover from, and learn from operational disruptions. And it means, sometimes, to even implement a multicloud strategy, which enables businesses to better withstand the next threat to business continuity and prepare for the multiplicity of unknowns as we progress.”
Yuval referred to the Solarwinds and Colonial Pipelines cyber incidents, which remind us that we should never settle in our efforts to be prepared for such cyber threats in our organizations. Threat actors will continue to evolve with greater complexities and strike with greater impact.
We need to constantly be learning from incidents (internally and externally) even if they don't directly impact us. Yuval revealed a continuous exercise that he has implemented within the bank, known as the Near-Miss exercise.
“Running a Near-Miss exercise based on recent incidents (although we were not affected) has significantly helped us to identify areas of improvement and be better prepared for the next attack. These areas of improvement might relate to a missing action in our playbooks, a weak control coverage, a broken process, missing or not up-to-date policy, and more. This is a great opportunity to take a proactive action to uplift the resilience of the organization. Combining these Near-Miss exercises with continuous crisis management simulations will build a more resilient and cyber-ready organization.”
Yuval adds, “In addition, ensuring a Resilience by Design approach starts by frequently planning and simulating the crisis and incident responses internally, with our critical third parties, and regularly identifying improvement opportunities. This is crucial to defending against new threats without compromising client experience and system availability. Balancing a user-friendly experience with the complex requirements of enhanced security practices is critical.”
He concludes, “To be future fit, we need to adopt a ‘thrive’ mindset that recognizes that disruption is continuous rather than episodic and embraces disruption as a catalyst to drive the organization forward. In that, it's critical to remember that humans and teams led by humans can bring the degree of courage, judgment, and flexibility that is required in a dynamic environment such as we are witnessing and will for some time to come. To that end, let's focus on strengthening the ‘human firewall’ through training and awareness from the board down to the frontline, while increasing communications with our clients to keep them abreast, allowing them to stay vigilant against the fast-evolving cyber threat landscape.”
Preston D. Miller, CISO at NASA's Jet Propulsion Laboratory, has a great hybrid of perspectives. Previously, Preston was the Cyber Risk Information Assurance Manager and Incident Response Team Lead at the Washington Headquarter Services with the Department of Defense. Prior to that, he served in the U.S. Army. At NASA, applying the lens of cyber risk to the world of scientists and space, he had to adapt his thinking accordingly.
Coming from the Pentagon, they supported the warfighters, and “C – Confidentiality” was at the top of the triad “CIA.” “Can we relay messages and assure confidentiality around those messages and operations?” By ensuring they had encryption and operational security, everything they were implementing leaned toward confidentiality.
When Preston transitioned to NASA, he quickly realized that security was done differently there. “I ran into a bit of a culture shock,” Preston explained in a Mega C-Suite Stories19 recording. “The main business of NASA is to share information, science, and data with our external partners, to build spacecraft to land on planets like Mars; overall, to just advance the community's understanding of the universe and the solar system, which is a very different objective from supporting the warfighters.
“In my world at NASA, ‘I – Integrity’ is at the top of our list. It's not so much that we are trying to keep the data that we are sharing safe, but can we trust the data that we are sharing back and forth between our communities, and can we trust the data that we are sending back and forth to our spacecraft and operations?”
Preston had to adopt an entirely different mentality of how NASA defines risk, its primary concerns from a cyber risk perspective, and how to communicate with the business leaders around cyber risk. This is first about understanding the risk thresholds and the risk appetite of the core business units, which involves sitting down with the business leaders, system engineers, and project managers of space operations to learn what they care about and what they identify as risk to their mission. Then he must figure out a way to communicate risk in the same language. “One of the things I found out earlier on that tripped me up is that I had to work out some of those language barriers; what the business meant by significant risk is very different from what I meant by significant risk.”
Preston had to change his conversation and convert the risk for one particular business leader: “That old platform that you're using in one of the legacy space operations has a direct tie to this mission-critical system. And if we take those vulnerabilities that we found there, here's how an attacker can exploit those things to get access to that mission critical system. That can mean mission failure for you.”
Preston explained how he has been on a campaign to change the cybersecurity office from the office of “no” to the office of “know.” “Are we informing our end users of the right security principles, processes and procedures? Are we the office that they can rely on as a strategic partner at designing secure operations and systems? We want to give you the tools, the information, to design secure spacecraft and systems because we want to be an enabler to your mission success.” That comes with building trust, which creates an open space of having that conversation.
With NASA spending more than half a billion dollars on its space exploration efforts, it is important to be able to restore critical operations in a timely fashion. Having a robust disaster recovery plan and a carefully designed BCP is extremely important. At a minimum, it should be done on a yearly basis. However, for the mission-critical applications that would significantly impact the business, you might run your BCP more regularly, perhaps every quarter, even if it's just a tabletop exercise, to make sure the business-critical applications and services are well-tested and resilient. “Having a muscle memory for your business admin to know what to do in case of an event is really good.”
Preston's final piece of advice? “We may know our craft inside and out, but that does us no good if we can't tie that back to make the business successful. Be ready to listen to our business partners. People don't care how much you know until they know how much you care. Show that you care about their business, you care about their success, and I think that will pay many dividends for you and your endeavors as a security professional.”
In general, what does a good BCP look like for most businesses?
As part of their training and exercise programs, there are some companies that have engaged professional media consultants/TV reporters to role-play with their spokespeople and train them in articulating crisis issues. This will help you gauge the quality and effectiveness of your training. The nonappointed spokespeople should also be trained to direct the media to the spokespeople who have been equipped specifically to answer questions.
Finally, board education needs to include cybersecurity awareness. Some board associations (e.g., the North American College for Corporate Directors and the American College for Corporate Directors, who both provide education and credentials for boards of directors) have now incorporated cybersecurity training into their certification programs.
Practice makes perfect. Once the BCP is completed, the next step is to rehearse it and test it. Doing a walk-through tabletop exercise with everyone who is involved in the execution of that plan is incredibly important.
Legendary U.S. football coach Vince Lombardi once said, “Practice does not make perfect. Only perfect practice makes perfect.” A perfect practice of your BCP requires the involvement of your executives and the board of directors in the actual exercise. Boards and executives across all levels should recognize that their organization is a target and they need to be prepared to respond fast and well in times of crisis. When boards are invested themselves, putting their hand up to engage in these exercises, there will be less doubt about their roles should the business be breached, or appear on the news headline.
As a result, the organization's ability to recover quickly is strengthened. The most mature companies are those that invest the time to run these exercises, provide expectation training from the top down, and are committed to improving their BCPs with continual drills.
Several CISOs shared their views of the frequency of these drills, and all highlighted that short and weekly drills are valuable for the technical team. This ensures that the staff maintains their qualification and sharpens their skills and ability on performing their tasks in accordance with the different crisis scenarios, and within a specified timeframe.
For a much broader operational testing, the security operations center (SOC), for example, is worth running SOC testing at least once a month, and even perhaps once a week if a company has the bandwidth.
For CxOs and the board, running a tabletop exercise annually with the crisis action team is crucial. Companies that are high on the learning curve generally do them twice a year.
We close this chapter with a brief story that was sent to us by Arden Peterkin, who manages a complex K–12 environment for one of the largest U.S. school districts, with over 100,000 endpoints. Arden is the district's information security officer, and he urged us to highlight the ability to automatically respond to many security incidents in real time.
Arden's security team acquired a security orchestration automation and response platform (SOAR) capability, internally referred to as the “Robot,” to prepare for what they believed would be increasingly persistent ransomware attacks. The robot was programmed to provide 24/7/365 monitoring of their data centers and endpoint systems. It would eliminate the dependency on human intervention to diagnose and respond to critical security events in a timely manner. And, because of the short interval between infection and ransom demand, the robot was programmed to act on perceived threats within three minutes or less.
This approach has been very successful for the school district. They have had 16 observable ransomware occurrences that were mitigated without human intervention, within three minutes, with no noticeable impact. In almost all cases, the events were triggered by phishing emails. Thankfully, they have had few to no instances of false detections or isolations due to ransomware's unique characteristics.
It is because of these and other cyber defense strategies currently in place that the school district is prepared to sustain, monitor, and improve their cybersecurity posture during these challenging times.