WILLIAM B. EIMICKE
In previous chapters we presented cases and analysis to illustrate how the elements of social value investing—process, people, place, and portfolio—contribute to the success of cross-sector partnerships. This chapter focuses on the fifth and final element of the social investing framework, a partnership’s performance. To illustrate how a focus on performance measurement can produce outcomes that meaningfully impact people’s lives, we present several examples from New York City. Over the course of a few decades, New York City transformed from one of the most dangerous cities with the least effective city government in the world to one of the most admired and copied. New York City adopted performance management citywide in 1977, but the effects were limited until 1994 when Police Commissioner Bill Bratton created and implemented the CompStat system, a performance management program focused on crime prevention.1 CompStat has evolved over the past several decades, but it continues to help keep New Yorkers safe. By the turn of the century, Mayor Michael Bloomberg directed the enhancement of CompStat and the successful application of similar, performance-based management systems for the city’s emergency and rescue services, operated by the Fire Department of New York City (FDNY). Over time, FDNY’s focus on performance measurement—ranging from response times and lives saved to fires prevented—as well as new performance management systems, led to a transformational change in how it accomplished its mission and served the public.
Everything about performance management in New York City changed on January 10, 1994, when a new mayor, a self-identified crime-fighter named Rudolph Giuliani appointed William J. Bratton commissioner of the New York City Police Department (NYPD). Bratton had a track record of reducing crime in Boston and in the New York City subway system.
Bratton had extraordinary success in transforming New York City from one of the most violent and dangerous big cities in the world into one of the safest, with a multifaceted strategy using techniques that had helped other cities. Bratton’s major contribution to policing theory was elegantly simple—policing should be focused on crime prevention first. Before Bratton, NYPD was famous around the world as fearless, relentless, and, occasionally, corrupt. Bratton changed the NYPD for decades to come by harnessing the big data the agency was already collecting and making it available in real time so precinct commanders could predict emerging patterns of crime geographically and focus their resources on “hot spots” before criminals could take over a neighborhood.2
The system started with a few dedicated in-house believers, particularly the late Jack Maple,3 appointed by Bratton as a deputy commissioner for designing the CompStat crime control strategies. In the beginning, CompStat used only elementary desktop computers, a big map of the city, and some colored pushpins. Over time, the NYPD’s data and analytics program evolved into a partnership with IBM to create the NYPD Real Time Crime Center and the program known as CompStat, which the city still uses today.
Using FBI’s Uniform Crime Reporting categories of serious crimes,4 CompStat provided weekly reports on the number of these crimes, precinct by precinct, across New York City’s seventy-seven precincts (each holding 100,000 or more residents). Precinct commanders, guided by the top brass at headquarters, can deploy their resources to most effectively control and ultimately reduce crime. Bratton chaired weekly (twice weekly in the early years) citywide meetings to hold precinct commanders accountable for their performance. CompStat enabled top NYPD leaders to question lower-level commanders regularly and in person, in front of a large number of their supervisors and peers. Crime dropped by 12 percent in 1994 compared to 1993, whereas the comparable figure nationwide was a drop of 1.1 percent.5 By the end of the decade, citywide crime was down by 50 percent.6 CompStat subsequently won the prestigious and highly selective Harvard University Kennedy School Innovations in American Government award and has been copied by cities across the country and around the world.7
Based on the extraordinary success of CompStat, the Giuliani administration tried to bring CompStat-style data-driven, predictive performance management to other city agencies—with limited success. FireMarc, the FDNY version of CompStat, failed miserably. It languished without real commitment and suffered from a poor choice of indicators and few incentives or penalties—sticks and carrots—to ensure accountability. In the Department of Parks and Recreation, Parkstat fared somewhat better. Parkstat used inspections and a rating system to grade pass or fail equipment, benches, and leisure areas of each park. But here again resources were limited, and there was no effective accountability system.
The Central Park Conservancy (CPC) under the leadership of Doug Blonsky developed an effective system for parks, using CompStat-like zone management and performance standards and accountability, comprehensive manuals, standard operating procedures, and detailed handbooks for horticulture management, turf management, and trash management and recycling (see chapter 5). Partnering first with Mayor Michael Bloomberg and more recently with Mayor Bill de Blasio, CPC is now training City Parks Department managers to deploy the same accountability system in other city parks throughout New York City.
MAKING NEW YORK’S BRAVEST EVEN BETTER—FROM TACTICS TO STRATEGY
Businessman Michael Bloomberg faced daunting challenges when he became the first new mayor elected post-9/11. Major areas of lower Manhattan had been completely destroyed or severely damaged, the city’s economy was depressed, the government’s budget had a large and growing deficit, and even though crime was still dropping, the city faced a very real and imminent threat of terrorism as well. Bloomberg was committed to running the city like a business, which meant belt-tightening and fiscal discipline. But Bloomberg also understood that serving customers (in this case city residents and visitors) was at least as important. Bloomberg used performance-based accountability and partnerships to do both extremely well.
Despite the fiscal constraints, Bloomberg continued to invest in CompStat and the impact-based approach to policing, and he worked aggressively and effectively to bring CompStat-style performance management to other New York City agencies, particularly the FDNY.
FDNY, New York City’s fire and emergency medical services agency, has always been one of the world’s best. On September 11, 2001, FDNY personnel became America’s heroes as they evacuated 13,000 to 15,000 people8 from the two World Trade Center towers in the 102 minutes between American Airlines Flight 11 hitting the north tower and when the tower collapsed at 10:28 AM.9 This extraordinary success came at a very high price—343 FDNY personnel died,10 a staggering nearly one-third of the total number of 1,172 FDNY deaths from the very first firefighter to die while on duty in 1865.11
With so many deaths and the subsequent realization that FDNY initially had serious difficulty determining who was at the site and who was not, it was clear to the department and incoming mayor Bloomberg that FDNY needed to improve its systems management. The Mayor’s Office commissioned the well-respected consulting firm McKinsey & Company to conduct a pro bono management review of FDNY and NYPD, focused on how both agencies could be better prepared for future major events, whether it be terrorism, fire, epidemic, or natural disaster. McKinsey’s key recommendations included that FDNY should “improve accountability and discipline” through “enhanced planning and management.”12 To accomplish this, McKinsey recommended increasing the number of personnel in management positions, enhancing the planning and analytical skills of those managers, and focusing more on “explicit metrics and milestones” to monitor the overall performance of the FDNY.13
To implement the McKinsey Report recommendations, the FDNY recognized that the vast majority of its senior managers came from the field—that is, firefighting—and had little training in management, analysis, planning, and organizational communication. This leadership and management challenge was acute given FDNY’s size—an annual operating budget exceeding $1.8 billion, 1.7 million incidents annually, and a workforce of over seventeen thousand spread over 250 locations.14 To overcome this training gap, Mayor Bloomberg and Fire Commissioner Nicholas Scoppetta engaged Columbia University to partner with the FDNY to develop an executive leadership and management training certificate program, modeled on the core curriculum of Columbia’s MPA and MBA degrees.15
Every year since 2002, mixed cohorts of sixteen16 of the FDNY senior management (and those the department viewed as the next generation of leaders) from both the fire and EMS divisions have completed the FDNY Officers Management Institute (FOMI), including a program-long team project.17 Sessions are taught at GE’s state of the art John F. Welch Leadership Development Center, a 59 acre corporate university campus founded in 1956 and named after legendary former GE CEO Jack Welch. As one GE employee selected to attend a program at the Welch Center described the experience, “this is where you go to get promoted.”18
GE’s president and CEO at the time, Jeff Immelt, was one of the first people to contribute to 9/11 charities after the attacks on the World Trade Center, and from the very beginning Immelt and GE have been full partners in FOMI. GE has donated the full cost of housing, food, classrooms, support, and recreation for all FOMI participants and Columbia University and FDNY faculty since the beginning in the fall of 2002. Given the all-day, every-day nature of FDNY, getting managers off site to a full-service learning center is an essential element in the success of the program. Participants continue to be paid during the training time but are taken off line by the FDNY and live and study at the Welch Center, an hour train ride north of New York City. Immelt and other GE leaders have addressed every FOMI cohort. Receptions and meeting opportunities allow FOMI and GE managers in programs at the center to get to know each other and share best practices.
Mayor Bloomberg and Commissioner Scoppetta recognized that partnering with Columbia University and GE could teach practical management skills and problem-solving techniques that FDNY participants would use to significantly improve FDNY performance. Over the past fifteen years, the FDNY-Columbia-GE partnership has prepared well over two hundred participants for senior management positions, and most of the top jobs at FDNY have been filled by FOMI graduates for more than a decade. A FOMI certificate has almost become a requirement for promotion to senior management. The quality and effectiveness of FOMI has been recognized by Harvard University’s Innovations in American Government annual competition, selecting FOMI as a “Top 50” program in both 2008 and 2009.
FOMI has played a major part in bridging the divide between FDNY’s fire and EMS divisions.19 FDNY leadership credits FOMI’s mixed cohorts of the next generation of fire and EMS leaders with creating more of a unified FDNY culture; this has resulted in much safer and more effective emergency services agencies, particularly as the number of fires has declined over the past decade and medical and other emergency calls have increased dramatically. Today, it is more likely that FDNY will save a life through a medical intervention than a rescue from a burning building.
In 2004, FOMI graduate Chief Joseph Pfeifer led the effort to create the first formal FDNY Strategic Plan in the department’s then nearly 140-year history.20 Over the past decade, the plan dramatically increased the FDNY’s success in securing federal grants from the U.S. Department of Homeland Security and others for equipment, including new fireboats, advanced technology, and grants for natural disaster and many other types of training programs. The plan also led to major improvements in emergency response operations, health and safety initiatives benefiting FDNY members, increased diversity, enhanced fire prevention and education initiatives, and a much stronger and more effective management and organizational system. The Strategic Plans also led to a $17 million upgrade of the Fire Department Operations Center (FDOC) in 200621 and a new FDNY Terrorism and Disaster Preparedness Strategy to be implemented by Chief Pfeifer, who then became the director of the FDNY Center for Terrorism and Disaster Preparedness.22
FOMI created a stronger learning culture at FDNY. The department probably has the best-equipped emergency services training center of any U.S. city, and a wide array of educational programs operate year-round, primarily focused on tactics and field operations. FOMI has encouraged management degrees at major universities across the country including Harvard’s Kennedy School, the School of International and Public Affairs at Columbia University (SIPA) and the Naval Postgraduate School, and a certificate in Counter Terrorism from West Point. Chief Pfeifer now holds a Master’s in Public Administration from the Kennedy School and a Master’s in Security Studies from the Naval Postgraduate School.23 FOMI graduate Assistant Chief of Fire Prevention and Citywide Command Chief Richard Tobin, a leader of FDNY’s risk-based inspection innovation, went on to earn his MPA from SIPA. Several other FOMI graduates went on to earn degrees from the Naval Postgraduate School. Currently, FOMI graduate Battalion Chief for Times Square Mike Meyers is pursuing his Master’s in Public Administration at SIPA. Partners FDNY, Columbia University, and GE all benefit from the shared learning happening at the Welch Center among current and future leaders from the public (FDNY), private (GE), and nonprofit sectors (Columbia University); and the public benefits from this sharing of best practices, which enables the FDNY to better save and protect lives and property in New York City, today and for decades to come.
PREVENTING FIRES
Former fire commissioner Nicholas Scoppetta once said, “Inspections are not firefighters’ favorite activity” (firefighters carry hoses, an ax, someone in need of rescue, not a clipboard and pencil, or even an iPad).24 Those who relish running into burning buildings do not enjoy walking around stores and office buildings issuing violations for overloaded electrical sockets or bicycles cluttering hallways. Firefighters previously disliked inspection activities enough that their unions negotiated contracts to limit inspection periods to two days a week and only when weather conditions were moderate.
The importance of effective inspections (figure 11.1) for saving lives became very clear in the aftermath of a major fire at the vacant Deutsche Bank building in Lower Manhattan (adjacent to the World Trade Center site) on August 18, 2007, which took the lives of two firefighters. The building was severely damaged on 9/11 but was still under slow deconstruction because of the presence of asbestos, toxic materials, and possibly human remains. Firefighters went into the building to put out the fire because of the environmental dangers so close to nearby occupied buildings.25
Figure 11.1 A Fire Department of New York firefighter carrying out an inspection before the new prevention-based priority process was developed by the NYC FDNY-IBM cross-sector partnership. Photo by Richard Numeroff, courtesy of SIPA Case Collection.
The investigations that followed the incident discovered that a test of a standpipe that had been noted as breached earlier that year was never completed, and an even larger break in the basement was completely missed.26 The fallen firefighters were unable to escape because there was no water coming from the cut standpipe to fight the fire, and the primary escape route down the staircase was improperly blocked. Fire officers at several levels failed to carry out the rule that buildings under construction or demolition must be inspected every fifteen days. Inspectors at the Department of Buildings also failed to properly train inspectors and conduct proper inspections, and they did not communicate important information to other city agencies, including the FDNY.27 FDNY executives were forced to confront the reality that in 2007 inspection records were not digitized in easily accessible databases. Rather, many were kept on index cards, written in pencil, and stored in local FDNY captain’s firehouse offices (of which there were over two hundred and twenty). Accessing inspection information or monitoring the inspection program from FDNY headquarters was virtually impossible.28
A team of FDNY members, civilian and uniformed, were charged by the Mayor’s Office with developing a comprehensive solution to the obviously flawed inspection system. The team quickly concluded that the decentralized, paper-based program was outdated; cyclical inspections in such a large city were wasteful and ineffective; and prioritizing by catastrophic risk (such as schools and hospitals) had led to multiple annual inspections of the city’s safest buildings and infrequent inspections of buildings most likely to have a fire, explosion, or structural collapse. With a workload of three hundred thousand buildings, fewer than four hundred civilian inspectors, and limited inspection time for uniformed firefighters, the team concluded that a radical reform of the current system was in order. However, the city government did not have the in-house knowledge of technology and global best practices to build a new system on its own.29 After careful consideration of multiple options, FDNY turned to IBM (NYPD’s long-time partner on the Real Time Crime Center and on CompStat) for help.
Claudia Gerola, an experienced IBM business strategy and development consultant, and a team from IBM worked closely with the FDNY team to map out the inspection process and interview the people who actually did the work, from the firefighters on the truck, to the captain in the firehouse, to the IT system managers, and throughout headquarters to the decision makers who needed the information.30 The two teams conducted a gap analysis to determine what data were needed to build a predictive inspection prioritization model, what data were already being collected, and what data needed to be collected or found. They explored solutions to close those gaps and then began to build a system with the primary goal of preventing as many fires and building-based emergencies as possible—for fires that did occur, they hoped to make them less dangerous and damaging.
As FDNY Deputy Chief Richard Tobin (figure 11.2) described it,
We saw all of the successes the police department had with CompStat. We saw where they were targeting their resources to where the crime was occurring. The whole idea was to get there before the crime occurred, saturate the area. And we wanted to duplicate the same thing with our inspection process. We didn’t want to wait for a fire to hit, we wanted to be out there proactively inspecting these buildings, eliminating their hazards before they had a fire there.31
Figure 11.2 FDNY Chief Richard Tobin reviews paper files with previously inaccessible data for setting inspection priorities. Photo by Richard Numeroff, courtesy of SIPA Case Collection.
The FDNY needed a system like CompStat that could analyze all the inspection information available (from FDNY and from other city, state, and federal agencies) and use a learning algorithm to prioritize the most dangerous buildings in their area for the inspection teams for each inspection period. That way, even with limited inspection resources, the opportunity to prevent damage and death before it happened would be greatest.
The first step in creating a new system was to digitize the existing inspection database. Then, to create a process for collecting new inspection information digitally, and to identify and track key performance indicators tied directly to program success. Digitizing the data required close cooperation among IBM, FDNY headquarters, the FDNY IT unit, and the firehouses. It was slow and challenging because firefighters in the field were not particularly interested in inspections in the first place. But some firehouses were more interested than others, and the team picked them to pilot the system. As the new system began to take shape, cooperation across the field operations increased. By 2009, there was a computer in every firehouse—replacing the index cards and pencils of 2007—connected to the FDNY Metrotech Center headquarters in Brooklyn, so senior management could review, analyze, and monitor the entire inspection program for the first time.32 As Commissioner Scoppetta said in 2009 after the Deutsche Bank fire investigations, “I think that if there is a silver lining to this cloud, it is that there is not an officer or a firefighter in the FDNY who does not recognize how important inspections are, how it saves lives and sometimes it will be the lives of our own members responding.”33
By 2010, FDNY and IBM were testing a new system that provided captains in the pilot-tested neighborhoods with a list of the most dangerous buildings in their area in real time at the beginning of each inspection period. The local captain could override recommendations based on experience or local knowledge, but the variation in buildings inspected would have to be explained and entered in the computer database. Pilot-testing made clear that a truly predictive inspection system would require data from multiple city agencies: the Department of Buildings, the Department of Environmental Protection, the Finance Department, Sanitation Department, Health Department, and even NYPD. Violations issued by these other agencies could also indicate a higher risk of fire. Failure to pay taxes could indicate neglected building maintenance. Police activity in a building might indicate dangerous chemicals being used in making illegal drugs or explosives used by terrorists. State and federal agency databases could provide similar information to make the predictive model even more robust.
Mayor Bloomberg created a citywide fire response task force almost immediately after the Deutsche Bank incident. Directed by Caswell Holloway IV of the Mayor’s Office (one of Mayor Bloomberg’s most trusted advisors, particularly in crisis situations),34 the task force identified the interagency nature of risk-based inspections and initiated the data sharing that would make the new system predictive. Although most city agencies were well on the way to digitizing their data, most used different platforms and many were using outdated legacy equipment. As Chief Tobin told us, “Sharing that data across [agency] lines was very difficult, really difficult.”35 Fortunately, the mayor was working on a parallel track to share and publish as much city data as possible through his commitment to open government and citizen engagement.
Beginning in December 2009, Mayor Bloomberg charged a few young analysts led by a charismatic, hard-driving former federal lawyer named Mike Flowers (who had worked on the Saddam Hussein trial in Iraq) to find a way to share data that could mitigate city problems before they spun out of control. Flowers knew the city had a tremendous amount of publicly accessible but agency-siloed data that could be better used to improve the quality and efficiency of government services. Flowers also knew that the more than forty city agencies could not change their systems, procedures, platforms, and identifiers for years, even if they wanted to (which they didn’t). Even if there was a will, he and his team of six did not have access to the money and technology, much less the time, to get it done. In any case, Bloomberg wanted results—and he wanted them yesterday.
Rather than go to war with agency heads, his team, dubbed the “Geek Squad,”36 recognized that technology had advanced so that data from different platforms and systems could be synthesized for problem solving in their present form. With this insight, Flowers and team set up the system that incorporated information from the Buildings Department, Department of Finance, and other agencies into the FDNY platform, which improved the accuracy in identifying the most dangerous buildings so FDNY could inspect those buildings first. In May 2013, the FDNY Risk-Based Assessment System was officially launched. During its development and after its official launch, the number of serious violations issued by FDNY inspectors has risen significantly, and fatal fires since 2009 have held at historically low numbers.37
Flowers and the Geek Squad went on to other data-matching triumphs, including identifying and preventing mortgage fraud, uncovering Medicaid fraud to access OxyContin and oxycodone, and tracking down restaurants that were illegally dumping cooking oil into city sewers. Harvard professor and former mayor Stephen Goldsmith would later describe them as “the best city data team in the United States, and one of the best in the world.”38 Goldsmith served as a deputy mayor under Mayor Bloomberg and was a major force in establishing the Mayor’s Office of Data Analytics (MODA), led by Flowers, and in leading Bloomberg’s efforts to share 311 and other city data with the public, organized geographically, so citizens could find out easily what the city was doing in their neighborhood, good and bad.39
Reflecting on his experience as the senior New York City official overseeing the risk-based fire inspection innovation, Goldsmith described it as an iterative process. FDNY was hampered by an underpowered internal technology system and limited staff support, but it had the on-the-ground personnel. IBM brought vast processing capacity, but perhaps too broad of an approach. And the Geek Squad brought knowledge of the entire New York City data system with insight on what could be accomplished rather quickly using cross-agency collaboration and the ever-increasing power of algorithms. This led Goldsmith to initiate a major innovation of his own—creating teams of Building Department and FDNY inspectors to work jointly on buildings identified by the algorithm as presenting the greatest risk of fire.40
The evolving partnership of FDNY, IBM, New York City Department of Buildings, and the Mayor’s Office illustrates how rethinking a program—fire inspections—to deliver quantitatively better results was made possible through a cross-sector, multiorganization partnership. Accessing the more sophisticated technology and analytic skills of IBM and City Hall’s Geek Squad enabled the application of advanced algorithms to actually prevent fires and building-related emergencies. Given the chronic and long-standing underfunding of IT equipment and staff in most city government agencies, this innovative breakthrough would not have happened without a partnership focused on performance measurement and analysis.
“IN GOD WE TRUST. EVERYONE ELSE, BRING DATA.”41
Mike Bloomberg became one of the richest people in the world by figuring out how to get the best information to Wall Street traders and managers before anyone else realized how important and profitable that might be. For twelve years, from 2002 through 2013, he used data-driven accountability systems to make the New York City government more effective than it had ever been. He often chose partnerships to accomplish complex challenges, sometimes private companies such as IBM and GE, sometimes nonprofits, and sometimes philanthropic organizations, including his own foundation. (He used his foundation to facilitate implementation of some of his most creative and risky innovations because he did not want to experiment with taxpayer dollars, such as the Rikers Island Social Impact Bond program described in chapter 10). His reliance on partnerships and performance contributed significantly to his overall success; in 2013, an independent panel of historians and political scientists rated Bloomberg second only to Fiorello La Guardia as the best mayor in the city’s very long history dating back to 1665 (and several rated him first, even though he was still in his third term at the time).42
BLOOMBERG’S LEGACY FOR SOCIAL IMPACT MEASUREMENT AND MANAGEMENT
Through Bloomberg’s performance-based approach, his administration partnered with a wide range of large and small for-profit and nonprofit organizations to drive improved outcomes and an ever-better city for twelve years.43 Many of these examples proved that impact-based partnerships can make a real, positive difference in people’s lives. While limits to performance measurement in its current practice remain, it has great potential for improving public programs and services, as highlighted by the comprehensive management programs in New York City.44
Data-driven performance management did not originate with Mike Bloomberg; in fact, CompStat began under Rudy Giuliani, but Bloomberg recognized that city activities could be dramatically improved by following a CompStat-style performance-based model. The process: first, carefully think through the mission and goals (prevent crime and prevent fires rather than catch criminals or put out fires); second, designate key impact indicators that enable senior managers to measure how well their organizations are doing in achieving their mission (cut serious crimes in half or eliminate fire-related deaths); and third, hold all members accountable for success.
At a general level, this approach to performance management is applicable to a broad range of social missions. In chapter 12 we discuss a variety of impact measurement tools and outline how a predictive based methodology might apply to many of the cross-sector partnerships profiled in this book. We explore specific questions such as: How might the FDNY score new fire prevention programs and choose which to fund? We also detail the new Impact Rate of Return (iRR) formula, which helps partners compare potential program options or strategies to achieve a desired positive social impact.
