CHAPTER 1
A Data-Driven Approach to Urban Science and Policy
IN 1996, BALTIMORE, MARYLAND, introduced the first 311 hotline. 1 It arrived with little fanfare or anticipation of its future influence. Rather, the goal was to solve a relatively mundane practical issue. Inner-city Baltimore was suffering from high levels of crime and blight, and the city was receiving enough reports for shooting and other serious events that calls about “nuisances,” such as graffiti, abandoned buildings, and other issues of deterioration, were themselves seen as a nuisance. The 311 hotline was thus born of a need to triage 911 calls that did not qualify as emergencies.
It was not until a decade later that the advent of digital technology made apparent an additional advantage of 311: Equipped with the information from resident reports, operations departments could generate automated work-order queues that guided the daily deployment of resources. This enhanced the value of 311 systems for major metropolises, but it also raised the possibility that they could make government services more effective and efficient for municipalities of any size. As a result, 311 hotlines and allied programs are now in place in over 400 American municipalities in 40 states and counting, spanning the geographic and demographic range of the country. 2 Since then, 311 systems have become a de facto symbol for the field of urban informatics. They have inspired blog posts and magazine articles, including the widely distributed Wired essay “What 100 Million Calls for Service Can Tell Us about New York City.” 3 Publications focused on governance have either trumpeted the benefits of 311 outright or coyly posed questions such as, “Is the cost of 311 systems worth the price of knowing?” (coming to the eventual conclusion that, “Yes, they are.”). 4 They have stimulated research projects, including our flagship project at the Boston Area Research Initiative (BARI), which forms the main content of this book and has given rise to methodological and philosophical approaches that guide much of our other work.
The 311 systems have proliferated quickly, but, given that there are plenty of other technological innovations in cities that merit attention, why have they become so emblematic of urban informatics? I would argue that it is because, in addition to their widespread popularity, 311 systems embody each of five major themes whose convergence characterizes the field. The first two themes form the bases of the field: (1) the innovative use of novel data resources and (2) the utilization of crowdsourcing and sensor technologies that provide a detailed view of patterns and conditions across the city. Their value has been amplified by (3) widespread data sharing, or, in its most extreme case, “open data.” This has been a critical mechanism for supporting a civic data ecosystem in which individuals and institutions from a range of disciplines and sectors can pursue and collaborate on questions of common interest. Finally, these collaborations have been channeled into two main, and often complementary, products of the field, which constitute the fourth and fifth themes: (4) technocratic policy innovations that improve the efficiency and effectiveness of city services, and (5) the scientific pursuit of a deeper understanding of the city and its people, places, and systems. Importantly, this view highlights two lessons that are often overlooked, especially by “smart cities” narratives. First, the products of modern data and technology need not be immensely expensive or flashy to be both informative and useful. Second, cross-sector collaborations are critical in generating these products.
Part II of this book presents an overview of the field of urban informatics, using 311 to illustrate how modern digital data can catalyze cross-sector research that generates both new knowledge and public value. This first chapter articulates and details the five main themes of the field, describing how 311 reflects each. In addition, because urban informatics is a young field and thus still relatively small, it is possible for me to summarize in this chapter many of the primary research programs that compose it. I do not provide a stand-alone list of these programs but instead describe various examples throughout the chapter in order to capture the five themes in action while also giving the reader a sense of the range of models for this work. 5 I will go into some depth on BARI, discussing why 311 has acted as the jumping-off point for us. Whereas the current chapter emphasizes the inspiration and potential of the field, Chapter 2 will follow with a more critical assessment of how one properly conducts research with the novel digital data resources that form the bases of urban informatics, again using 311 to demonstrate both the challenges and the possibilities.
The Bases of Urban Informatics
New Technology, New Data
At its foundation, urban informatics has emerged from recent advances in digital data and technology. These resources have generated new information that I divide into two forms for the purpose of presentation: enhanced forms of old information, and novel information produced by new technologies. In the first, the digitization of many administrative processes that previously existed only on paper has given rise to numerous data sets that capture the patterns of the city in intricate detail. This is occurring across the public, private, and nonprofit sectors, with examples ranging from credit card purchases, to rides on public transit, to entries to community centers; the tracking of energy and water usage to yearly vehicle inspections; the marriage registry, business licenses, building permits, tax assessments, and restaurant inspections; and, of course, requests for public services through 311 systems. This list is far from exhaustive, but it gives a sense of the diverse range of data generated by the individuals and institutions of the city. All told, their digitization makes newly accessible a wealth of information on the behaviors, movements, social interactions, commerce and industry, and physical and environmental conditions of the city.
As digitization increases the potential utility of administrative records, two other technologies are generating entirely new kinds of information. The first of these technologies builds off of social media and other internet sites and applications that gather user-generated content, also known as Web 2.0. The content shared with these platforms—Yelp! reviews, Picasa pictures, YouTube videos, exercise and sleep activity from FitBit bracelets, “tweets” through Twitter—are data that one might organize, map, analyze, and interpret. A subset of Web 2.0 applications also supports direct communication between a client and a service provider, be it private or public, capturing every transaction as a data record. This capacity has taken hold in 311 as well. Boston introduced Citizens Connect (now BOS:311) in 2008 as an early effort to introduce a smartphone app for a municipal 311 system, leveraging the internet and smartphones as an additional channel for constituents to request government services. Other cities have since followed suit.
The second technological advance of note is the proliferation of sensors. Some examples include GPS trackers for geographic mobility patterns; accelerometers that detect different types of physical movement; and sensors that record the density of pollutants in air and water, ambient temperature, light intensity, precipitation, noise levels, or physical vibrations. Some “sensors” we might not even think of as such. For example, wi-fi hot spots can be used to estimate pedestrian traffic by counting the number of devices that engage them. New image-processing programs translate footage from security cameras into estimates of pedestrian, bicycle, car, and truck volume through a space. Many cities have also deployed “shot spotters” that detect the sound profile of gunshots. These are just a few examples, but they serve to illustrate the broad potential of sensor technologies.
A Composite View of the City
The knowledge derived from modern administrative data, sensor technologies, and Web 2.0 applications evokes an approach to measurement that combines many narrow observations to build a comprehensive view of the world. This is not an entirely novel concept—for example, Sampson, Raudenbush, and Earls developed a methodology in which they surveyed thousands of Chicagoans about their neighborhood to create robust measures of physical and social conditions across the city—but its scale and generality in urban informatics is distinctive. 6 In the case of sensor technologies, a city or research center might deploy a set of units that track local conditions in real time. Each observes only a small slice of the world, but their composite provides detailed coverage across space. For example, the University of Chicago’s Urban Center for Computation and Data, which I will discuss further in the next section, is deploying a system of sensors called the Array of Things in Chicago. The sensors track localized environmental and atmospheric conditions and activity, and the overall system is billed as a “fitness tracker for the city.” 7
In Web 2.0, human users provide the individual pieces of information. This process is referred to as crowdsourcing, a term that has entered common parlance through efforts such as Wikipedia, where the many members of the “crowd” collectively contribute to knowledge. At the intersection of crowdsourcing and sensor technologies is citizen sensing, in which members of the public are either an active or a passive vehicle for observing and recording events and conditions. At the most passive, cell phone records register the location and activity of a user every time the user engages with a cell tower. On the other end of this spectrum, in one project bus drivers voluntarily carried GPS trackers in an effort to identify the unofficial “routes” of Nairobi’s informal transit system. 8
Administrative records offer a third way to gain a composite view of the city. At times, these may be classified as citizen sensing, as the information is provided by constituents submitting forms or requests. For example, one might argue that 311 encourages residents to act as “the eyes and ears of the city.” In turn, it crowdsources a constantly updating map of the potholes, streetlight outages, and downed trees that need attention. Other administrative data, such as tax assessments, are generated through internally directed processes. Whether they arise from citizen sensing or not, administrative processes, just like sensors, generate thousands or even millions of records, each describing a discrete event or condition at a specific place and time. In turn, their corpus can be aggregated to describe localized patterns.
The intertwined trends of (1) the emergence of novel data resources and (2) crowdsourcing and sensor technologies have provided a new view on the city. First, the data are diverse in their content, often capturing types of information that have never before been available. They are also endowed with unprecedented spatial and temporal precision, permitting extensive flexibility in their analysis and communication. Consequently, they are not limited to the sorts of annual indicators that are characteristic of traditional data sources—things like median income, ethnic composition, and rates of crime, poverty, and disease. Instead, they offer a window into the pulse of the city, or the daily rhythms and long-term trends of the places, people, and institutions that constitute an urban area.
Exposing the pulse of the city for observation, analysis, and interpretation has been a guiding inspiration for some of the oldest efforts in urban informatics. The term itself was originally coined by British planner Michael Batty in an essay of the same name. 9 Batty might be considered one of the founding fathers of urban informatics, having led the Bartlett Centre for Advanced Spatial Analysis (CASA) at University College London since its inception in 1995. It was ahead of its time, leveraging cutting-edge analytic and visualization tools to study and inform urban design and planning at a time when computer-based mapping was a brand-new technology. To this day, CASA remains at the forefront of the field, implementing advances in both methodology and theory, with a particular emphasis on complexity.
Another early leader in revealing the pulse of the city was Massachusetts Institute of Technology’s (MIT) Senseable Cities Lab (Senseable), founded by Carlo Ratti. As its name implies, Senseable has been notable for its pioneering use of sensor technologies and citizen sensing to illustrate the patterns of the city. Ratti is an architect by training, and he and his team often demonstrate their scientific insights through captivating visualizations and interactive displays that are regularly featured in museums and galleries. Notably, neither CASA nor Senseable have centered their work on a single region, reflecting an interest in urban science defined broadly rather than an in-depth effort to study a single city. As we will see in the next section, this contrasts with some of the models that have emerged in recent years.
The Pulse of the City: Toward a Computational Urban Science
On the one hand, being able to access the pulse of the city would appear to hold much promise. On the other hand, because it is literally the daily patterns we each know and experience laid bare, initial work centered on it can sometimes seem obvious. I think of an article written by members of Senseable that I teach each year that shows that cell phone data usage in business districts in Hong Kong, London, and New York City differs markedly from that of residential neighborhoods in the same city across weekday, weeknight, and weekend hours, capturing the familiar shifts in activity between work and home. 10 A common reaction among the students is, “Well, of course.” Indeed, we know that most people start weekdays at home, head into work, and then come home in the evening, with an extended respite on Saturdays and Sundays. It is by no means revolutionary to discover that the geographical locus of their cell phone usage reflects this rhythm.
As reasonable as the students’ skepticism might be, a devil’s advocate might retort: “Have you ever seen this information represented with such empirical precision?” The value of the work lies not in a fundamental discovery about the city but instead in its novel methodology and the future discoveries it promises. Instrumentation and recordings are the basis of science and its fundamental goal of building a cumulative, organized body of knowledge. Take the example of the movements of the celestial bodies. Humans surely have been paying attention to and attributing meaning to them for tens of thousands of years. Some of these rhythms are clearly visible to the naked eye: over the course of about 28 days, a full moon will gradually wane until it vanishes completely, only to wax anew; day length peaks at the beginning of the warmest period of the year and is at its shortest at the beginning of the winter. The comprehension of more complex phenomena, however, requires detailed record keeping. Johannes Kepler, for example, used extensive charts on the movements of the skies to represent the orbits of the planets with formal mathematical equations, forming the basis of our understanding of the mechanics of the solar system. In turn, these equations have since been critical to everything from space travel to the inference that there is an unseen Planet X lurking beyond Pluto. 11 Similar breakthroughs in observation are visible in myriad domains of science. Microscopy opened up the world of cells and the study of the building blocks of life. Atomic spectroscopy permitted the identification of the representation of atoms of different elements, allowing chemists to directly observe the composition of molecules. The identification of DNA and the elements of its code has given rise to the fields of genomics and genetic engineering. The list goes on, but in each case, the ability to observe and record a particular phenomenon opens up a whole new area of inquiry.
David Lazer and his colleagues have argued that the instrumentation of society with sensors and other digitized processes will be similarly revolutionary, giving rise to a computational social science that reaches across subdisciplines, including urban science. 12 Thus, knowing that cell phone usage is greater in business districts on weekdays between the hours of 9 am and 5 pm is just the tip of the iceberg. It is simply a validation exercise demonstrating that such patterns are indeed visible through cell phone records. What lies beneath these seemingly mundane observations is the ability to query the data to answer any number of more complicated questions. Do these patterns change during a rainy day, a snowstorm, a parade, or the aftermath of a terrorist attack? Does the profile of usage in a particular region of a city portend future events, such as the emergence of a new industry, the collapse of an existing one, or the gentrification of a neighborhood? The patterns might be disaggregated to the person level as well, permitting us to ask questions about individual differences in movement and communication. For example, are they different depending on a person’s socioeconomic status or the industry she works in? What proportion of people does not follow the daily routine of morning-workday-evening or has no discernible rhythm at all? Where are people with these irregular patterns most concentrated? In sum, by recording the pulse of the city in intricate detail, digital technologies and data create an opportunity to know not only how it looks and operates today but also to predict and manage how it might evolve tomorrow.
Emerging in the earliest days of urban informatics, work by CASA and Senseable has not only exposed the pulse of the city but also demonstrated the potential knowledge and utility it might provide. The work of CASA pays special attention to the complexity and consequences of urban form. For example, it has used historical street maps to model how streets appear, lengthen, or become segmented over time. 13 It has also analyzed how urban form can influence energy efficiency, offering implications for the relationship between urbanization and climate change. 14 Senseable has lived up to its name through a variety of projects, including deploying GPS trackers in trash items to uncover major inefficiencies in how they are transported through the sanitation system. 15 It has also used a year of taxi rides in New York City to identify routes used commonly enough to justify ride-sharing programs. 16 In a project called Underworlds, it has placed sensors in the sewer system in hopes of detecting disease and infection at early stages with fine geographical precision. Altogether, these projects constitute first steps toward a computational urban science, or, as we now know it, urban informatics.
311 and the Pulse of the City
The 311 system offers a composite view of certain urban dynamics and, in turn, access to a slice of the pulse of the city. Hundreds of requests for
service stream into the system daily, capturing the needs of communities in real time (as can be seen in the YouTube video at https://
That said, there remain some unaddressed methodological issues regarding the interpretation of different case types (e.g., does a request for a special trash pickup for furniture contain the same meaning as one for removal of graffiti?) and whether the volume of calls reflects the density of issues across the city or the density of individuals who choose to report them. Such concerns exist for all of these novel data sets and will be examined more closely in Chapter 2 . For the moment, the point is simply that 311, like many of the other newly available data sources, uses the actions of urbanites to crowdsource the localized needs and conditions, thereby providing one vantage point on the pulse of the city.
FIGURE 1.1 The pulse of the city of Boston as seen through 311 reports, including (a) differences in reporting volume on weekdays and weekends, (b) the increase in 311 activity in February thanks to snow removal requests, (c) the spike in pothole requests in March, and (d) the sharp drop in requests during the Marathon bombings (Monday) and the ensuing manhunt (Friday).
Data Sharing and Collaboration: Catalyzing the Field
“Open data” has become a buzz term and even part of the common vernacular in recent years. Generally speaking, it refers to data that a government has made publicly available, though in reality it can come from any organization willing to share data with few or no restrictions for access. One might attribute the movement toward open data to a variety of actors. There are activist groups, such as the Sunlight Foundation, that have called for the release of data as part of a broader push for government transparency. Such groups argue that these data belong to the public and that Freedom of Information Act requests create an unnecessary burden for access when it would be just as easy for the government to upload the data for all to see. 18 For its part, government has become more than just a reluctant follower in this trend, recognizing that valuable insights and tools might arise if the broader community of analysts and “hackers” gain access to these data. Private companies, such as Socrata, have also seen a market opportunity, building software for platforms that specialize in hosting data and documentation. In this manner, open data has come to offer potential benefits across sectors.
Like other aspects of the digital revolution, open data is a societal shift that has been magnified in urban areas. Many cities have passed “open data ordinances” that require departments to publish their data in machine-readable formats (i.e., spreadsheets that can be analyzed). In New York City, for example, part of the IT budget for every department is contingent on compliance with this dictum. A large number of cities have also contracted with Socrata or other vendors to implement platforms that facilitate public access to the data. “Hackathons,” or whole-day events in which analysts compete to create the most compelling analyses, visualizations, and tools based on open data, are now commonplace. Indeed, 311 requests are often one of the first data sets to be released on a city’s open data platform, as they were in Boston.
Though open data receives the bulk of attention, it is actually just one part of a broader trend toward data sharing, motivated largely by the potential for collaboration. Indeed, many municipal open data ordinances are as much about sharing data between departments—to better inform and integrate management across city operations—as they are about sharing data with the public. This was the explicit intent behind the ordinances of New York and Boston, the former resulting in the centralized DataBridge, which gathers and coordinates the data from the city’s many agencies and departments. Cities have also embraced data sharing across institutions and sectors in order to support projects that are either more targeted than the crowdsourced approach of a hackathon or require data sets that are too sensitive to publish through an open data platform. This was the case for an effort in Massachusetts that has lowered the legal hurdles to access for researchers to health and human services data regarding the opioid crisis in the hope of stimulating new innovations in prevention and treatment. When such cross-sector collaborations are built on a shared investment in transforming the data into insights and tools, they promise mutual benefits, and because the data in question are often updated regularly, they open the door for a sustained relationship between the collaborators, which has become a major mechanism for driving successful urban informatics projects.
From Data Sharing to Partnerships
To better understand how digital data sharing can catalyze research-policy collaborations, let us imagine a lunch meeting between an academic and policymaker during which they discover a number of common interests. As they begin to plan a study, the academic says something like, “I’ll need to find funding for a graduate student. We need to write and validate a survey, and then to administer the survey and organize and analyze the data. This will all take about three years.” The policymaker might respond with some version of, “Thanks for thinking this through, but I have a newspaper headline from yesterday that needs my attention, so I’m not sure this works on my timeline.” Clearly, this is a caricature, but the point stands: talks break down as a result of different institutional incentives and timelines.
Now consider how the conversation changes with the advent of large-scale digital administrative data. The academic might instead say, “You have 300,000 calls for service in that database. Give me and my graduate student a month or so; we’ll do some first-cut analyses and write you a memo. We can then regroup and figure out the next steps.” Suddenly, a project that produces information of common interest while also fitting the timelines of both parties becomes possible. These origins lead to a project with a form that is distinct from the one-off projects that often characterize research-policy collaborations. Though not a hard rule, traditional models for collaborations between academics and policymakers take one of two forms: either a researcher requests data from an agency to write a paper, or the agency contracts with a researcher to conduct a particular analysis, often a program evaluation. Even when successful, such arrangements typically do not lead to a sustained relationship. Rather, the form of collaboration I describe here entails an iterative process in which the answers to the initial questions inspire the next phase of work. Put somewhat glibly, urban informatics is driven by “partnerships not projects.”
My primary focus here and throughout the book is on partnerships between cities and universities, but the same dynamic might easily involve any combination of participants, including public agencies, nonprofits, private companies, and university researchers. In addition, public agencies need not be the source of data in every case, being that each of these entities generates data. Whatever the specific nature of the project and the roles of the participating parties, digital data sharing creates the potential for a sustained partnership that grows and evolves with each new question, discovery, and innovation. Furthermore, it means that there may be multiple overlapping partnerships going on in a given city at any time, creating a network of data sharing and collaboration that is greater than the sum of its parts. This might be referred to as the civic data ecosystem.
The Civic Data Ecosystem
To ground the term “civic data ecosystem” more formally, let us use a relatively standard definition of the word ecosystem from biology: the species of organisms that inhabit a space, the physical characteristics of that space, and the interactions among them. In the current case, the space in question is not so much a physical one as it is an informational one, and the fundamental resources, much like sunlight and water for a forest, are data and data-generating technologies. In place of “species of organisms,” there are different types of institutions—public, private, nonprofit, and academic—each of which contributes to the ecosystem in a characteristic way. Some generate greater amounts of data, others are particularly skilled in data management and analysis, while others are best positioned to interpret and communicate findings. Activities and interactions across these institutions then determine the products that emerge: the insights gained, the technologies advanced, the solutions proposed. And, just as biologists recognize that organisms regularly alter the physical environment of their ecosystem, the products of these institutions will continue to shape the informational context and the activities that might thrive within it. 19
Casting urban informatics work as occurring within and through a data ecosystem presents a distinctive view of the field. Most importantly, it highlights collaboration as the primary mechanism for translating data into products. Without collaboration, everyone is simply analyzing their own data for their own isolated purposes, creating a collection of narrow insights that do not necessarily intersect. By extension, as the teams undertaking these collaborations become increasingly inclusive, so do the products, often answering questions that are at once relevant to scholars, policymakers, and the public. Partners might even work on projects that intentionally construct and evolve the ecosystem itself; for example, cohosting workshops or hackathons can initiate desired projects, but they also support further community building. It is in acknowledgment of this spirit of collective effort and public-minded data science that I refer to this not only as the data ecosystem but also as the civic data ecosystem of the city, and why this might be thought of as the second urban commons in this book.
The efforts of three urban informatics programs—New York University’s Center for Urban Science and Progress (CUSP), the University of Chicago’s Urban Center for Computation and Data (CCD), and Northeastern and Harvard Universities’ Boston Area Research Initiative (BARI)—have embodied the civic data ecosystem mindset. In each case, an academic institution has partnered closely with an active and forward-thinking city government to gain a comprehensive understanding of that city and to develop innovative solutions to its challenges. A major component of these partnerships between city and university is the amassment and integration of data from diverse sources, in turn constructing an extensive resource that might support any number of projects. Additionally, the university partner in each case has launched an educational program in urban informatics to train a new generation of scholars and public servants who are experts in the skills and concepts that constitute the field.
In New York, for example, CUSP has aligned itself closely with the Mayor’s Office for Data Analytics (MODA), managing a parallel feed of the city’s DataBridge and cosponsoring a number of fellowships, with students working for MODA and city employees studying at CUSP. Urban CCD has worked with the city of Chicago on a range of projects, including the development of predictive analytic models that forecast events of major interest, from shootings to rat infestations, and the development and deployment of the Array of Things sensor system. The BARI program is similar to NYU CUSP and Urban CCD in structure and purpose but is distinctive in being an interuniversity effort that convenes faculty from the region’s many institutions of higher learning. This creates an even more extensive and diverse network of collaboration.
It is worth noting that the number of centers that seek to catalyze urban informatics work within a single city has grown considerably in recent years, embodied by centers such as Metro21 at Carnegie Mellon University in Pittsburgh, Pennsylvania, 21st Century Cities Initiative at Johns Hopkins University in Baltimore, Maryland, and the nonprofit Envision Charlotte in Charlotte, North Carolina. Notably, most of these other centers do not themselves conduct data science, administer educational programs, or create policy but instead act primarily as conveners and connectors for those who do. For this reason, I group these only loosely with NYU CUSP, Urban CCD, and BARI.
Boston’s Civic Data Ecosystem
The interuniversity nature of BARI goes back to its origins. In 2011, two professors of sociology at Harvard University, Robert J. Sampson and Christopher Winship, approached two institutes within the university, the Radcliffe Institute for Advanced Study and the Rappaport Institute for Greater Boston, and the city of Boston with the concept for a Harvard-Boston Research Initiative. The project was made feasible by the fact that this was more the continuation of a conversation than a completely new one. The executive director of the Rappaport Institute, David Luberoff, had long overseen multiple collaborative efforts with the city of Boston, including a highly successful summer fellowship program that embedded graduate students in public agencies. He and Christopher Winship had also had previous discussions over the years with colleagues at the city of Boston about a broader collaborative framework, though there had never before been the resources and political will to see it through. In keeping with a major theme here, though, these various activities had created trust between the institutions, laying the fertile ground that made the newly proposed initiative possible.
The Radcliffe Institute committed initial funding for the project, and shortly thereafter I was hired as a postdoctoral researcher to oversee it. In the ensuing months, we realized that the initiative needed to be inclusive if it were to fully realize its mission. The Boston area is home to the highest density of elite academic institutions in the world, housing a wealth of talent that would be key to a truly comprehensive urban research agenda. With this in mind, we hosted a symposium titled “Reimagining the City-University Connection” at Radcliffe. The unwritten agreement was that if we managed to attract at least 100 people there was sufficient enthusiasm to justify moving forward. Final counts estimated about 420 attendees.
With that, BARI was born (pronounced as in the “a” in “marker,” in order to distinguish us from a prominent colleague named Barry) with the mandate of not only undertaking research projects that advance both science and policy but also fostering the collaborative relationships between researchers, policymakers, and practitioners that form the backbone of a thriving civic data ecosystem. To achieve these goals, BARI has three main areas of activity: (1) pursue core research-policy partnerships that focus on major themes and challenges facing greater Boston and at the forefront of urban science; (2) develop technologies that make emergent data sources accessible for research, policy, and practice, centered around BARI’s flagship project, the Boston Data Portal, a public platform where researchers, policymakers, and community members can map and download data generated by BARI projects; and (3) convene and support cutting-edge research and policy work in the region through various mechanisms, including an annual conference, 20 workshops, a web site and network powered by LinkedIn, and seed funding for graduate student projects.
Given the richness of Boston’s civic data ecosystem, we have had the good fortune to partner closely with a variety of institutions from across the public, private, nonprofit, and academic sectors, many of which will appear as collaborators in the ensuing chapters. For the moment, it is worth naming just a few to capture the different approaches to urban informatics that exist in Boston and to hint at the potential advances that lie at their intersection. Northeastern University’s School of Public Policy and Urban Affairs (where I am on the faculty) has long been home to the Dukakis Center for Urban and Regional Policy, a “think and do tank” dedicated to applied research that informs effective policies. More recently, Northeastern University launched a master of science in urban informatics, a program that merges modern data science with a substantive understanding of the dynamics and challenges of the city. Through BARI, the students have the opportunity to learn through Boston-based data and collaborations with local leaders. At Boston University, the Initiative on Cities, founded by former Boston mayor Thomas M. Menino, looks both globally and locally to identify ways for researchers and policymakers to partner in the development of essential services and sustainable infrastructure. Meanwhile, both the Boston Civic Media Consortium (based at Emerson College’s Engagement Lab) and URBAN.Boston (based at the University of Massachusetts Boston) are dedicated to pursuing research projects in collaboration with local community groups while also incorporating the voices of everyday Bostonians into the conversation around data science. Notably, each of these groups takes its own distinct approach to urban informatics, creating a context well suited to collaborations and projects of various forms.
On the policy side, BARI’s closest partners have been the city of Boston’s Mayor’s Office of New Urban Mechanics (MONUM) and Department of Innovation and Technology (DoIT). MONUM is a unique entity that has played an important role in the advancement and popularization of urban informatics both locally and nationally. Its self-described role is to be an incubator for innovative approaches to improving city services and government-constituent engagement, or what it refers to as “the City’s R&D team.” This often entails technological solutions, but not as a rule. Boston DoIT has played the complementary role of building out the data systems and analytics team necessary to support innovations of this sort. Similar efforts have also been under way at two bordering cities, Cambridge and Somerville, and, more recently, at the Commonwealth of Massachusetts. In addition, the regional planning agency, the Metropolitan Area Planning Council, has a highly active data services department, exploring as well how such work might be extended to smaller municipalities that do not have the resources to develop such technologies themselves. In the nonprofit sector, The Boston Foundation’s Boston Indicators project has sought to use data to better tell the story of Boston’s past, present, and future and to identify issues in need of attention. Finally, in the private sector, there has been active engagement from some of the local technology companies, notably Microsoft New England’s Technology & Civic Engagement arm.
When BARI began in 2011, the 311 system presented itself as a fitting pilot project for testing the potential for collaborations on research and policy. MONUM had recently introduced the smartphone application Citizens Connect to augment the Mayor’s Hotline (as 311 in Boston was then known), and there was much interest both locally and nationally as to how this experiment would work out. Additionally, the system was already storing all requests in state-of-the-art databases that were relatively clean and research-ready. At this point, the simple vignette I described earlier (“Let us spend a month looking at those 300,000 requests for government service and we can start to figure out what questions we might answer”) really did happen. In the ensuing five years, it went from a pilot project to something quite a bit more expansive, incorporating Boston DoIT; Boston About Results, the city’s performance management team; the Public Works Department; and a cadre of academics from Harvard University, Northeastern University, and Emerson College. In sum, it has been an effective proof of concept, and a clear manifestation of the latent potential we believed we saw in Boston’s civic data ecosystem after that first symposium at the Radcliffe Institute.
Products of the Field
Thus far, I have summarized the first three themes of urban informatics: (1) novel data resources and (2) crowdsourcing and sensor technologies, which form the bases of the field, and (3) cross-sector data-sharing partnerships as a critical mechanism for seeing the work through. What remains to be discussed are the products of urban informatics projects. Products are important because they represent the tangible results of the work but also because they reflect the motivations of the various contributors. The final two themes summarize the products of the field: technocratic policy innovations that utilize the insights and tools of urban informatics to improve city services, and the scientific pursuit of a deeper understanding of the city. One might be tempted to segregate these two elements, treating the former as indicative of “smart cities” and the latter representing “the new urban science,” but this would fail to capture their mutually reinforcing relationship. New innovations depend on the conceptual advances of scholarly research, and scholarly research develops new questions based on the pressing challenges of the real world. Examining the nature of these products and their interactions will enable us to better understand the purpose and goals that energize the field and its constituents.
An important point as this section moves forward is the range of forms that the products of urban informatics can take. Of course, there are the flashy innovations that captivate the popular imagination, such as autonomous vehicles and ubiquitous sensor systems, but these do not reflect the vast majority of the everyday impacts that modern digital data and technology can have. More importantly, these futuristic innovations are only within reach for a small proportion of cities worldwide and thus fail to provide the generalized promise of the field. With this in mind, I make certain to include numerous illustrative examples that use data sources available in most if not all municipalities. In turn, this highlights how urban informatics promises insights and implications that are broadly accessible today while also laying the groundwork for the cities of tomorrow.
Innovating on Policy and Practice
As has been noted, many city governments, especially those in major metropolises, have embraced the potential of data and technology to make services, programs, and other aspects of governance more efficient and effective. Goldsmith and Crawford presaged the transformation that such innovations would bring, but rapid growth in the field since the publication of their book merits an additional summary, albeit brief, of the current state of these efforts. 21 For the sake of organization, one might loosely categorize the tools that have emerged from this work into three groups. First, there are efforts to deploy Web 2.0 applications and sensors to be the “eyes and ears of the city,” capturing local conditions and patterns. The 311 system epitomizes this potential for Web 2.0, especially when a city implements internet portals and smartphone applications for reporting. Boston was a leader in this effort to leverage Web 2.0 for government services with the Citizens Connect app, but a diverse range of cities have followed suit, from New York City to Tuscaloosa, Alabama. The potential value of sensors is captured by examples such as the effort by Pittsburgh, Pennsylvania, to equip municipal vehicles with sensors that indicate when they have hit potholes, thereby saving Public Works the trouble of searching for them.
A second type of innovation has been the development of platforms, often dubbed “dashboards,” that combine multiple data sets to track conditions and performance. These have their origins in New York City’s CompStat program, started in the 1990s, in which the police department mapped out crimes and arrests to better grasp the trends in each precinct, and a similar cross-agency effort called CitiStat in Baltimore early in the following decade. The modern manifestations of this approach benefit from a notable expansion in data resources and statistical tools. In Chicago, for example, the city has built the WindyGrid (the public version is called OpenGrid), 22 which maps data based on custom-defined events and time periods (e.g., homicides in July, streetlight outages in 2015), representing hot spots and cool spots that had a greater or lower density of events than would be expected based on historical data and other localized characteristics. Another approach has been Boston’s CityScore, which reports major indicators, such as the number of shootings and the percentage of work requests filled on time, through a public interface. 23
A third type of innovation combines the power of the two others to create new programs and policies. These are in some ways the most powerful endorsements of the societal value of urban informatics while also being the hardest to identify and point to. Whereas the two others entail platforms and tools that can easily be trumpeted in newspapers and blogs, the impact of this third set of innovations can only be seen over time, and even then it might be difficult to communicate. There are many different examples of such programs, only a few of which I will be able to list here.
- The city of South Bend, Indiana, and the University of Notre Dame reengineered the sewer system to eliminate a major problem with sewage backflow entering people’s houses during times of peak water flow. More recently, a company that emerged from this project equipped the sewer system with sensors to identify blockages in real time. 24
- Chicago, Santa Cruz, Los Angeles, and other cities use predictive analytic models to forecast where major crimes are likely to occur, informing policing strategies. 25
- A complementary set of programs in Boston 26 and Minneapolis 27 have used data to identify and target “problem properties” that generate an inordinate amount of crime and disorder.
- New York City has aggressively used data to drive policies surrounding transportation. They used GPS trackers on taxis to inform and then evaluate the reorganization of Times Square to be more pedestrian friendly. They also used detailed data on collisions between cars and bicycles to inform policies to limit such events. 28
There have also been efforts to extend this sort of work into the world of health and human services, which has data that is more sensitive and problems that do not lend themselves to formulaic solutions. Though such work is less prevalent, it has gained momentum in recent years. An early leader in this area has been Actionable Intelligence for Social Policy, an initiative based at the University of Pennsylvania and centered on the construction and utilization of integrated data systems (IDSs). These link data from multiple health and human services agencies at the individual level, providing unprecedented opportunities to analyze patterns of service use, consequences of traumatic events, 29 and programs and policies that coordinate data from various sources. Examples include:
- Homelessness policy across the nation, including Philadelphia, has increasingly utilized data to distinguish between “chronic” and “crisis” homeless and target services appropriately. 30
- In some cases, the mere access to data is the innovation. For example, Medicare workers in South Carolina have a system that uses many pieces of information about an elderly patient, including those already entered in the medical system and others entered by the worker, to suggest the necessary level of care. This permits more flexible treatment plans, with the hope of allowing the elderly to remain in their homes (rather than in nursing facilities) for as long as possible. 31
- In Allegheny County, Pennsylvania, home of Pittsburgh, an IDS enabled an algorithm that could predict the level of risk for child abuse for a given case. This has been implemented to direct resources at the call center to cases that are more likely to lead to serious issues. 32
Importantly, technology and data alone do not necessarily make for great policies and programs, and there is clearly a need for evaluation. Nonetheless, these and other examples illustrate the potential of digital data and technology to craft new approaches to urban governance and services.
All of the efforts listed here were realized through research-policy partnerships, with academics participating in various ways, from developing metrics, to building statistical models, to helping guide program design. One of the institutions seeking to expand this collaborative civic problem solving across cities has been the MetroLab Network (MetroLab), a consortium of city-university partnerships focused on bringing data, analytics, and innovation to city government in order to benefit local communities. MetroLab was launched by the White House’s Office of Science and Technology Policy, with the city of Pittsburgh and Carnegie Mellon University agreeing to sponsor the initial phase (it is now an independent 501c3 organization). Founding members included Boston-BARI, Chicago–Urban CCD, and New York City–CUSP, as well as a number of cities that are smaller or are less prominent in this space, such as South Bend, Indiana–Notre Dame and Memphis, Tennessee–University of Memphis. MetroLab’s effort to expand such work nationally is twofold. One part of its work is to support additional cities to develop the cross-sector partnerships that underpin urban informatics. The second is to construct a network that enables the learning and transfer of innovations from one city to another, thereby spreading the associated benefits more widely.
The Policy Vision for Urban Informatics
With the number of technocratic innovations in cities growing rapidly, one might ask what the overarching philosophy for this work is. In his book on smart cities, Anthony Townsend argued that rather than exotify such innovations, we should recognize that they are just modern solutions to the same problems we have always faced—sanitation, transportation, infrastructure maintenance, education, and public safety. 33 This is true on a surface level, but it does not necessarily mean that policymakers engaged in urban informatics are approaching the problems with a single-minded focus on the objective improvement of operations. The 311 system, for example, does help government to deliver services more efficiently and effectively, but it has received just as much attention for the new forms of interaction it creates between government and the public. It has been heralded as a democratizing force, making city services more accessible and responsive, while also encouraging constituents to participate directly in the governance of their own city. Thus, though technology may provide new tools for old problems, the ways in which policymakers design and implement them also reflects other societal trends.
The 311 system is not the only case in which municipal governments have leveraged technology to become closer to the people they represent and serve. In fact, the phenomenon is widespread enough to have its own name: civic tech. Civic tech takes many forms, most of which use Web 2.0 sites and applications to enable public deliberation and discourse. For example, the Community Plan-It platform uses an interactive virtual environment to elicit residents’ ideas on development decisions in their community. 34 Participatory budgeting has become increasingly popular, especially in Latin America, giving certain constituencies more say in the use of public funds. 35 Many dashboards are explicitly pitched as public platforms so citizens can track government performance and neighborhood conditions. In this manner, civic tech reflects a reimagining of coproduction, an approach to public administration that seeks to directly involve the public in the governance process. Whereas coproduction has traditionally been most visible in the form of parent-teacher organizations and community policing, technology has allowed it to take on new forms and enter new domains. Part III of this book will delve more deeply into civic tech and coproduction, but for the moment the primary point is that efforts by city governments to solve well-known problems by utilizing data and technology could take on a variety of guises. The current case has been centered on a civic spirit, which in turn is building a model for public involvement in government in the digital age.
The New Urban Science: In Search of a Paradigm
Whereas the policymakers involved in urban informatics are focused on improving the efficiency of government while also making it more accessible and participatory, the goals of their scientific counterparts are not always as clear. The fundamental role of science is to advance knowledge, and it is on the strength of those contributions that academics evaluate each other. Of course, those who are collaborating with policymakers and practitioners want their research to have societal impact, but the nature of the research itself depends on the body of knowledge they hope to advance. The broad interdisciplinarity of urban informatics, however, makes it difficult to pinpoint a particular “body of knowledge” that acts as the field’s primary focus. On the one hand, many of the “usual suspects” of urban science, such as sociologists, criminologists, public health researchers, and planners, have adopted modern data and methodologies as a way to further old questions about the city. At the same time, newcomers from other disciplines, such as mathematics, physics, chemistry, biology, and computer science, have been attracted to urban informatics by the opportunity to apply their computational skills and models to the complexities of societal dynamics. Consequently, the intellectual breadth of the field is wide ranging.
The diverse—or, some might say, fragmented—composition of the field poses a challenge. I am no proponent of narrow disciplinary orthodoxy (I am a biologist by training who has since worked in departments of psychology, sociology, public policy, and criminology), but there is something to be said for having a canon that at least partially unifies scholarly efforts. Without that, the result is not a collective project, which all science inherently needs to be, but rather a handful of individuals asking disparate questions about topics whose connections are not fully articulated or agreed on. Under these conditions, cumulative knowledge will be hard to come by. The great philosopher of science Thomas Kuhn argued that science is built on paradigms, or “a framework of concepts, results, and procedures within which subsequent work is structured.” 36 Put another way, a paradigm codifies the overarching theoretical questions that are of greatest importance, the methods for probing them, and thus a blueprint for the research that might ensue. The question, then, is: What paradigm or paradigms are guiding scientific inquiry in urban informatics?
A Comparison of Two Paradigms
Let us apply the Kuhnian perspective to the most basic question that an urban scientist might ask: What is the city? While this question might seem exceedingly simple, how one answers it will determine what he or she deems worth asking next, thereby setting the pathway forward. There have been many answers to this question over 150 years of urban science, but here I will compare two that are most prominent in the field of urban informatics. First is the approach to urban science developed by members of the Chicago School of Sociology in the early twentieth century, which emphasizes the social organization within and between neighborhoods. 37 Second is the “social reactor” theory recently proposed by a team of physicists at Santa Fe Institute and their colleagues. 38 These examples are also instructive in that they capture an underlying tension in the field between the extension of old ideas and the adoption of new ones.
In the 1920s, Robert E. Park and Ernest W. Burgess, working at the inception of the University of Chicago’s department of sociology, published a series of essays titled The City. 39 One of the foundational premises of their work was that the city was not a monolithic entity but rather one composed of many distinct communities, each with its own social organization—that is, formal and informal relationships, norms, and patterns of interaction. Writing at a time when industrial cities were rapidly growing, they saw neighborhoods as a social unit similar to a village, with a social organization grounded by personal relationships and localized institutions. The city, however, created a conglomeration of these many communities, requiring what Park and Burgess referred to as a secondary, or impersonal, set of institutions that could serve and operate a municipality of hundreds of thousands or even millions of people. This perspective has since inspired a number of lines of research, most notably work in sociology, criminology, public health, and others that emphasize the role of neighborhood social dynamics in shaping the mental and physical well-being of residents, thereby explaining the stark variation in outcomes we observe across the urban landscape. 40 Much of the theoretical approach to neighborhood dynamics used in this book also grows out of the Chicago School.
More recently, a group of scholars based at the Santa Fe Institute (SFI), led by Geoffrey West, Luis Bettencourt, and José Lobo, have argued that the city is a “social reactor,” within which increased population density results in greater interaction between individuals, in turn elevating overall productivity. They have formalized this argument using conceptual and mathematical models from biology and physics, revealing what they refer to as the universal scaling laws that relate city size to its outcomes. Empirically, they find that indeed the residents of cities with larger populations have a greater number of social connections, higher incomes, and are more productive, generating, for example, more patents. 41
The Chicago School and SFI perspectives are similar in that they each treat the city as a social system with special properties that merit attention, but their interests are at different scales. The SFI team’s social reactor hypothesis concerns itself with the processes and outcomes of an entire metropolitan area, including suburbs, positing very little about the events and conditions that differentiate neighborhoods. The Chicago School’s work instead attempts to answer this latter question by focusing on the dynamics and consequences of the social organization of neighborhoods. Consequently, these two conceptions of the city inspire different lines of questioning and as such two distinct “urban sciences.”
The Challenge of a Unifying Theory
Debate will naturally arise any time there are multiple guiding perspectives to the study of a broad topic. This is true even when the perspectives pose different questions of arguably equal importance, as in the case of the Chicago School’s emphasis on neighborhood social organization and SFI’s interest in metropolitan areas. A possible response is to privilege one perspective over the other as more important. For example, one could argue that because the social reactor hypothesis is rooted in computational methods and formal mathematical models from the “hard” sciences, it has greater merit. On the other hand, theory on social organization has remained influential for nearly a century, generating offshoots in numerous fields and having a sustained impact on our understanding of the city. Obviously, few would adhere wholeheartedly to either of these unilateral views of the field, but they are instructive because they do illustrate a broader tension within urban informatics between the traditional urban disciplines and the more recent arrivals.
Another solution is to develop a unifying theory of the city, but this is likely to be fraught with difficulties from the outset. A city is a stage for all aspects of human behavior and society, making it an ideal study site for just about any social phenomenon of interest. Furthermore, the real-world nature of the work facilitates interdisciplinary collaboration at the intersections of domains—for example, the simultaneous consideration of flow dynamics and human behavior when attempting to address traffic patterns—and offers a natural opportunity for the work to have direct public impact. Such a situation, however, does not lend itself well to a single intellectual framework. It would be difficult if not impossible to construct a single guiding paradigm that encompasses psychology, sociology, planning, political science, economics, education, public health, criminology, geography, and public policy, not to mention their respective subdisciplines and the components of biology, physics, chemistry, and mathematics that might also be incorporated into the conversation.
My intent is not to condemn urban informatics to be forever fragmented, a field lacking any canonical basis. Instead, the attitude that guides my own efforts and those of BARI is that there is much work to do before we get there, and that we will need to be satisfied if the “unifying theory of the city” is actually a series of interlocking paradigms, each of which sets the agenda for studying a particular class of phenomena. Returning to the foresight of Lazer and his colleagues regarding computational social science, the newly introduced data, methods, and perspectives will likely transform the way we think about many aspects of the city in the coming years, generating new theoretical perspectives and clarifying the intersections between them. 42 This will eventually result in a multifaceted synthesis that acknowledges the two paradigms I have presented here, a number of others that I have not mentioned, and, of course, a handful that are yet to be proposed. In keeping with this, BARI’s activities take a catholic view of the field, supporting projects across many domains, from the mapping of bicycle collisions from police narratives, to the use of historical census data to track shifts in demography, to the integration of administrative data and parent interviews in the evaluation of the public schools’ assignment system, to the detailed measurement of conditions that drive or mitigate the urban heat island effect, to digitizing Boston Public Library records to track how services are offered across neighborhoods. Our goal is to support the many disciplinary approaches to urban informatics and to create opportunities for interaction and cross-pollination across them. These are the necessary first steps to create the theoretical synthesis that promises to reshape and deepen our understanding of the city.
Integrating Urban Science and Policy
Given the distinct roles of policymakers and scientists, it is inevitable that they also differ in their motivations. The former are concerned with improving the efficiency and effectiveness of city services and programs, often with a flair for increasing interaction between the government and the public. The latter are broadly interested in contributing to our basic knowledge about the city, though the specific questions this entails depend on the researcher or program in question. These two sets of goals for urban informatics are distinct but complementary, and can be mutually reinforcing. In the one direction, nearly anything a researcher discovers about the city could, at least in theory, be relevant or useful to some agency or department. In the other direction, the current needs and trends of the city can ground research and give it a natural opportunity for impact.
The full opportunity for urban informatics to bridge the divide between research and policy, however, becomes clear in light of the other themes discussed in this chapter. First, new data resources and sensor technologies promise novel discoveries and tools, but it is not self-evident what these will be. Researchers and policymakers have a common interest in solving this riddle by developing techniques and approaches that can fully realize that potential. This symbiosis is especially apparent in the case of data generated by the administrative processes of public agencies, where the agency understands the origins and interpretation of the data, and a scientist can transform it into knowledge, much of which will be directly relevant to the agency’s operations and objectives. Second, digital data sharing can accelerate collaborative partnerships that might otherwise have fallen casualty to the differing timelines and incentives of academia and the public sector. Given the various motivations of each party, as well as the diverse set of questions the available data might answer, the resulting projects might take any number of forms.
To illustrate, 311 has been a catalyst for collaboration between researchers and policymakers because its data provide a number of avenues for study, which we might categorize into four groups. First, as captured in the main theme of this book, the data bear witness to behavioral dynamics that underlie neighborhood maintenance, or the custodianship of the urban commons. Second, 311 reports can be treated as “the eyes and ears of the city,” though this depends on methodologies that handle questions of measurement and validity—does the density of calls reflect the density of problems or the density of concerned individuals? Chapter 2 addresses this challenge for administrative data more generally with the test case of translating 311 reports into measures of physical disorder and deterioration (i.e., “broken windows”) across the city. Third, it is a tool for tracking the interactions between the government and the public, and for evaluating the effects of certain programs on this relationship. For example, does stop and frisk discourage constituent engagement with the government? 43 Fourth, it is the most prominent case for assessing the implications of “civic tech” and the broader move toward coproduction, a major focus of Part III of this book. Studies utilizing 311 in each of these ways have the potential to produce scientific discoveries that contribute to our understanding of cities. They also offer insights for those managing 311 systems and their colleagues.
The 311 system is one of many programs and data sets, each of which merits attention. Further, the combinations of these various sources of information can support and inspire an even greater array of mutually beneficial collaborations between researchers and policymakers. This might include anything from education, to transportation, to climate change, to gentrification, to resilience and security, just to name a few. It is not necessary that each party be interested in the exact same questions, just that there be sufficient overlap in the things that each would want to learn from a particular project. Thanks to the greater incentives provided by digital data and technology, and the lowered hurdles resulting from an ethos of data sharing, the likelihood of such work is far greater than it has been historically. This promises a truly synergistic urban informatics that has the dual goals of advancing both scholarship and policy.
Conclusion: The Introspective City
I began this chapter by listing the five themes of urban informatics, broken up into three groups. (1) Novel data resources and (2) crowdsourcing and sensor technologies form the bases of the field by providing access to the pulse of the city with unprecedented detail and precision. (3) Widespread data sharing acts as a critical mechanism for fostering a civic data ecosystem characterized by sustained collaboration across disciplines and sectors. These collaborations lead to two main types of products, (4) technocratic policy innovations that seek to improve the efficiency and effectiveness of city services and (5) the scientific pursuit of a deeper understanding of the city and its people, places, and systems. This book will capture each of these five themes, but, as a product in its own right, it embodies the reinforcing relationship between the last two. The empirical work on the urban commons is rooted in the fifth theme, translating a novel data set into substantive insights on the behavioral dynamics of neighborhoods, but it also demonstrates how these insights can be translated into the policy innovations captured in the fourth theme.
I want to close the chapter by considering what happens when these five elements come together. I see it as an exercise in introspection. Each research project is an effort by a city to get to know itself, bringing some aspect of its inner workings to light. As with all good introspection, this moment of observation is also an opportunity for action. In this case, the opportunity rests with policymakers, who might capitalize on the new knowledge by designing tools and practices that can improve the city for those who work, live, and play there. It is this cycle of discovery and improvement that makes modern behavioral therapies so effective for individuals, and, by analogy, it is what makes urban informatics more than just “smart cities” or a “new urban science” but instead an integration of the two. This, however, relies on the ability to appropriately leverage the novel digital data resources that have recently become available. This is a task that, like many in the world of “big data,” poses a distinct set of challenges, as we will explore in Chapter 2 .