GIS and the 2020 Census: Modernizing Official Statistics

The future is here. If you read this far, you are aware of the massive changes underway affecting GIS and statistics. The pace of adoption of new innovations such as ML and drones has been faster than expected. Some say that we are entering the human era of GIS. A modern GIS is about participation, sharing, and collaboration. It allows us to share information with anyone we choose and consume information we need that has been published by others. This chapter will expand on some of the ideas presented earlier on dissemination and spatial analytics and provide you with information about leveraging these and other new technologies in your organization.

According to the Pew Research Center, 65 percent of Americans search online for information about their government, yet less than 10 percent report finding what they need.¹ Citizens need and want information, and leaders need input from society. With data-driven citizenship as a top priority, ArcGIS Hub engages governments and communities around policy initiatives to tackle pressing issues. It is important to understand the issues your country identifies as significant, and equally important to have the input of citizens, civil society, and academia to help solve those issues. ArcGIS Hub brings top priorities into focus by helping to define initiatives and the data needed to better understand the issue. What is an initiative? We all need our local governments to take initiative to solve key critical issues. Initiatives usually are focused on a certain topic, such as decreasing traffic congestion, improving access to health care, or decreasing unemployment. These issues can be understood by combining different types of data and by using mapping and visualization tools to tell the story. Initiatives help support the overall goals of your government and focus on improving the lives of its citizens.

Initiatives work in ArcGIS Hub by combining data, visualization, and analytics with collaboration technology. Getting citizens to participate is often the key—by engaging and informing your citizens, organizations can achieve better involvement and therefore success. One good example comes from the city of Los Angeles called “Vision Zero.” The goal of this initiative is to reduce severe injuries and deaths in roadway collisions. Data needed to understand this issue includes basemap data, transportation network information, POIs (e.g., hospitals, retail centers, industrial centers), traffic data, and demographic data. Finally, information about the accidents themselves is needed—the type of vehicle involved, the location of the accident, the time of the accident, and the injuries or fatalities that were sustained. This data may come from various sources, including the Department of Transportation, the NSO, the NMA, or even local jurisdictions. The idea behind Vision Zero is that these deaths are both unacceptable and preventable, so the initiative strives to take a data-driven approach to reducing severe and fatal injuries.

This idea can also be applied to the SDGs. The goals as defined include targets and indicators that can guide initiatives. Imagine setting up an initiative for your country focused on reducing poverty in all its forms, everywhere. The data needed to understand many of the goals comes primarily from the NSO, though collaboration with other organizations is needed if we are to gain true understanding and take action.

Figure 11.2. Example of goals, targets, and indicators from the UNSD. Excerpted from a UNSD report on final goals, targets, and indicators. Source: UNSD.³

Indicators such as these all require basic demographic data, including disaggregations (e.g., by sex or age), income data, and location information at a minimum. In some cases, information from other local agencies such as the Ministry of Health or a disaster agency may be needed.

ArcGIS Hub allows any organization to tell compelling stories with data. By using story maps, dashboards, and infographic reports, data can be presented in intuitive ways. ArcGIS Hub allows any organization to share information, measure progress, and show accountability. It is the engagement focal point for top issues and a place to publish the work the community creates together.

As mentioned in an earlier chapter, UNSD has used hub technology to create the FIS4SDGs. This system allows for collaboration across the UN and with member states, creating transparency and empowering the community.

Hubs are just beginning to grow in popularity and are allowing nations to create a network of networks—bringing data from various agencies together to solve problems, innovate, and inspire.

As you have learned, hubs should tell compelling stories with data. Smart mapping, which was described earlier, allows just that: telling the story of the data. Smart mapping provides better initial parameters, such as colors, scale, and styling, that fit the data and map’s story. This functionality is why smart mapping benefits novices and experts, making both more productive.

Continuous color ramps and proportional symbols, improved categorical mapping, heat maps, and new kinds of bivariate maps that use transparency are delivered through a streamlined and updated user interface.

As technology continues to evolve, maps get even smarter. One of the newest features in ArcGIS is called ArcGIS^® Arcade expressions. Arcade is a portable, lightweight, and secure expression language. Like other expression languages, it can perform mathematical calculations, manipulate text, and evaluate logical statements. Arcade was designed specifically for creating custom visualizations and labeling expressions in the ArcGIS platform. It allows users to write, share, and execute custom expressions in ArcGIS Pro, ArcGIS^® Runtime, ArcGIS Online, and the ArcGIS^® API for JavaScript^™.

What makes Arcade particularly different from other expression and scripting languages is its inclusion of feature and geometry data types. Though one of Esri’s newest scripting languages, Arcade is not a full programming or scripting language for creating stand-alone apps; nor is it a replacement for automation. It is a focused, intuitive, JavaScript-like language for creating expressions that customize visualization and labeling. Think of it more like a spreadsheet formula.

With Arcade, calculations with layer fields can be easily performed, and the result for label expressions or data-driven visualizations can be easily used. This means that when making a map and the layer being used doesn’t contain the exact attribute field needed, data can be generated on the fly without editing source data, adding a field, or permanently calculating values. Simply put, Arcade expressions allow map-making from simple calculations, functions, data conversions, and brand-new representations of the data.

With Arcade, created expressions can be used without modification across the platform. For example, visualizations can be based off values returned from custom calculations in ArcGIS Pro and saved as web map items, and those custom visualizations can be shared for use in other web, desktop, and mobile applications.

Esri provides a public gallery of Arcade expressions to give some good ideas on how this works. This collection of maps is available as examples of Arcade expressions within the smart-mapping interface. These maps can be viewed along with the expressions used to create the cartography.⁴

Arcade is purposefully simple. Instead of the many programming constructs found in other languages, it has a rich library of data, logical, mathematical, geometry, date, and text functions that make it easy to do complex calculations.

We touched a bit on spatial analysis earlier in the book when we spoke about using analysis for optimizing EAs or for use in siting field offices. However, it’s a very broad topic. Spatial analysis is the process of examining the locations, attributes, and relationships of features in spatial data through overlay and other analytical techniques to address a question or gain useful knowledge. Spatial analysis extracts or creates new information from spatial data. More simply put, spatial analysis is how we understand our world—mapping where things are, how they relate, what it all means, and what actions to take.

Each of these categories has a set of related topics or questions, as shown in figure 11.5.

Many different tools are available that allow one to conduct spatial analysis. For example, ArcGIS^® Spatial Analyst^™ provides a rich set of spatial analysis and modeling tools for both raster (cell-based) and feature (vector) data.

A quick guide to spatial analysis

Understanding where

•Understanding where things are (location maps)

•Understanding where the variations and patterns in values are (comparative maps)

•Understanding where and when things change

Measuring size, shape, and distribution

•Calculating individual feature geometries

•Calculating geometries and distributions of feature collections

Determining how places are related

•Determining what is nearby or coincident

•Determining and summarizing what is within an area(s)

•Determining what is closest

•Determining what is visible from a given location(s)

•Determining overlapping relationships in space and time

Finding the best locations and paths

•Finding the best locations that satisfy a set of criteria

•Finding the best allocation of resources to geographic areas

•Finding the best route, path, or flow along a network

•Finding the best route, path, or corridor across open terrain

•Finding the best supply locations given known demand and a travel network

Detecting and quantifying patterns

•Where are the significant hot spots, anomalies, and outliers?

•What are the local, regional, and global spatial trends?

•Which features/pixels are similar, and how can they be grouped together?

•Are spatial patterns changing over time?

Figure 11.5. Categories of spatial analysis of spatial analysis (from the 2013 Esri workbook The Language of Spatial Analysis)⁵

The capabilities of Spatial Analyst are divided into categories or groups of related functionalities. This functionality can be accessed in several ways, including through a tool dialog box, Python^™, or a model. Traditional operations and workflows using map algebra or analysis using the Raster Calculator can also be performed.

Space-time analysis continues to advance as well. Creating a space-time cube allows one to visualize and analyze the spatiotemporal data in the form of time-series analyses, integrated spatial and temporal pattern analysis, and powerful 2D and 3D visualization techniques.

The ArcGIS Space Time Pattern Mining toolbox contains statistical tools for analyzing data distributions and patterns in the context of both space and time. ArcGIS allows the creation of space-time cubes and the conducting of emerging hot spot analysis.

The Emerging Hot Spot Analysis tool identifies trends in the data. It finds new, intensifying, diminishing, and sporadic hot and cold spots, for example. It takes as input a space-time Network Common Data Form (NetCDF) cube created using either the Create Space Time Cube By Aggregating Points tool or the Create Space Time Cube From Defined Locations tool. It then uses the conceptualization of spatial relationships values provided by the user to calculate the Getis-Ord Gi* statistic (Hot Spot Analysis tool) for each bin.⁶

Tools such as these can be used to detect and quantify patterns and make predictions. Many others are available that allow ArcGIS users to perform time-series clustering, outlier analysis, network analysis, or even 3D analysis. The power of spatial analysis does not stop there, and new developments in geospatial artificial intelligence (GeoAI) will continue to push the boundaries of what is possible.

To understand GeoAI, we need to first understand AI. AI has been defined as the “broadest way to think about advanced computer intelligence . . . anything from a computer program playing a game of chess to a voice-recognition system . . . interpreting and responding to speech.”⁷

According to the company McKinsey,⁸ AI is trending now owing to the convergence of algorithmic advances, data proliferation, and tremendous increases in computing power and storage, propelling AI from hype to reality. Think big data, IoT, and the cloud.

Another part of the equation here is ML. ML can be done in a supervised or unsupervised mode or by using something called re-enforcement learning. ArcGIS provides built-in ML tools such as clustering, classification, and prediction, bringing the “geo” to ML. According to Joseph Sirosh, corporate vice president of artificial intelligence and research for Microsoft Azure, “Integrating geography and location information with AI brings a powerful new dimension to understanding the world around us.”⁹ Further, “This has a wide range of applications in a variety of segments, including commercial, governmental, academic or not-for-profit. Geospatial AI provides robust tools for gathering, managing, analyzing and predicting from geographic and location-based data, and powerful visualization that can enable unique insights into the significance of such data.” Microsoft and Esri are partnering in this area,¹⁰ intending to bring AI, cloud, geospatial, and visualization aspects together in one application.

The areas of application of this type of technology seem to be almost limitless. One example might be in land change. What if you could predict land desertification based on the rate of change using models using data from historical imagery, weather, and tidal information? Or consider being able to predict agriculture and land efficiency from elevation, weather, crop yield, and soils information? The GeoAI, ML, and big data expert Mansour Raad tells us this is possible. Raad, who is the lead on advanced spatial analytics and the big data subject matter expert at Esri, challenges us to think about other areas of application, such as predictive incident analysis, and to consider the data needed for this type of analysis.

Think of the implications for this in official statistics. According to Raad, “When data volume swells beyond a human’s ability to discern the patterns in it—when a company is faced with truly big data—we need a new form of intelligence. GIS, infused with artificial intelligence, can help executives make better decisions.”¹¹

Whether it’s the use of data from mobile phones to understand movement or some of the new streaming data from the IoT world, statisticians and geographers both face the challenge of determining how best to apply this data and create meaningful information from it.

In a 2018 survey of global executives, Deloitte found that 65 percent of US organizations plan to invest more than $1 million in blockchain technology in 2019.¹² A blockchain is a growing list of records, called blocks, which are linked using cryptography. Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data. It is a ledger that records transactions in a verifiable, permanent way.¹³ If only 2 percent of all servers one day run on blockchain, Bank of America estimates that it will represent a $7 billion market.¹⁴ Geography is critically important to capture in a blockchain record, which is why we are now calling this a geoblockchain.

What do we as statisticians need to think about regarding geoblockchain? How will this new technology affect our work? Today, the answers are not clear, but what is certain is the potential for this new innovative technology. One example of the potential is an application by the UN World Food Program (WFP). The UN WFP blockchain system combines biometrics with blockchain technology to help those without an identification. For example, refugees, separated from their families, often have no public records or identification. It is difficult if not impossible to conduct normal daily transactions—such as buying food, withdrawing cash, or getting a credit card—with no identification. By combining biometrics with blockchain technology, WFP helps provide monetary aid to these displaced people; they save money on bank-related fees and can track daily transactions, just as they could with a credit card.¹⁵ This information can be used to better understand needs and inform other needed services.

Another example of a real-world application is from India, where blockchain is being used to make land registry more reliable. The idea here is to create a single source of truth of ownership status and a property history. The buyer is assured that the land being bought is the correct plot and that the seller is unequivocally the owner.¹⁶

Adding location to the blockchain would provide enhanced security and validation because the same transaction cannot happen in two places at the same time. Use cases for blockchain being explored today include land title, supply chain, and data exchanges. The amount of data that will become available with systems like these is worth consideration and needs research.

Understanding where

If you don’t know where you are, you are lost. Understanding where is about putting the world in context. Where are you? What is around you? Very similar to when you were two years old, your journey of spatial analysis requires an understanding of how you fit into your geography.

Understanding where includes geocoding your data, putting it on a map, and symbolizing it in ways that can help you visualize and understand your data. Within the taxonomy of spatial analysis, the first category of understanding where contains three types of questions.

TYPES

•Understanding where things are (location maps)

•Understanding where the variations and patterns in values are (comparative maps)

•Understanding where and when things change

Measuring size, shape, and distribution

The task of measuring size and shape is a common requirement in the spatial analysis process. You may want to know how large an object is, or you may want to describe an object in terms of its geometric properties, such as area, perimeter, length, height, and volume.

When there are multiple objects, the set of objects takes on additional properties, including extent, central tendency, and other characteristics that collectively define the distribution of the entire dataset.

The process of measuring and describing these characteristics constitutes the second category of spatial analysis questions.

TYPES

•Calculating individual feature geometries

•Calculating geometries and distributions of feature collections

Determining how places are related

Answering spatial questions often requires not only an understanding of context (understanding where), but also an understanding of the relationships between features. Take any two objects: How are they related in space?

How are they related in time? These relationships in space and time include associations such as proximity, coincidence, intersection, overlap, visibility, and accessibility.

Determining how places are related includes a set of questions that help describe and quantify the relationships between two or more features.

TYPES

•Determining what is nearby or coincident

•Determining and summarizing what is within an area(s)

•Determining what is closest

•Determining what is visible from a given location(s)

•Determining overlapping relationships in space and time

Finding the best locations and paths

A very common type of spatial analysis, and probably the one you are most familiar with, is optimization and finding the best of something. You might be looking for the best route to travel, the best path to ride a bicycle, the best corridor to build a pipeline, or the best location to site a new store.

Using multiple input variables or a set of decision criteria for finding the best locations and paths can help you make more informed decisions using your spatial data.

TYPES

•Finding the best locations that satisfy a set of criteria

•Finding the best allocation of resources to geographic areas

•Finding the best route, path, or flow along a network

•Finding the best route, path, or corridor across open terrain

•Finding the best supply locations given known demand and a travel network

Detecting and quantifying patterns

In the fifth category of the spatial analysis taxonomy, the keyword is patterns.

These spatial analysis questions go beyond visualization and human interpretation of data (from the understanding where category) to mathematically detecting and quantifying patterns in data. For example, spatial statistics can be used to find hot spots and outliers; data mining techniques can be used to find natural data clusters; and both approaches can be used to analyze changes in patterns over time.

TYPES

•Where are the significant hot spots, anomalies, and outliers?

•What are the local, regional, and global spatial trends?

•Which features/pixels are similar, and how can they be grouped together?

•Are spatial patterns changing over time?

Making predictions

The last category includes questions that use powerful modeling techniques to make predictions and aid understanding. These techniques can be used to predict and interpolate data values between sample points, find factors related to complex phenomena, and make predictions in the future or over new geographies. Many specialized modeling approaches also build on the physical, economic, and social sciences to predict how objects will interact, flow, and disperse.

Despite their differences, all these questions share the same principles: they are used to predict behavior and outcomes and to help us better understand our world.

TYPES

•Given a success case, identifying, ranking, and predicting similar locations

•Finding the factors that explain observed spatial patterns and making predictions

•Interpolating a continuous surface and trends from discrete sample observations

•Predicting how and where objects spatially interact (attraction and decay)

•Predicting how and where objects affect wave propagation

•Predicting where phenomena will move, flow, or spread

•Predicting what-if

This is an exciting and challenging time for those of us engaged in the data revolution. Technology continues to advance, and with it the potential to expand the use and application of statistical and geospatial data. The future is within our grasp, and data truly can help us understand the pressing questions of the present and the future. The more accessible that data is, the more important it will be to understand it. Maps are the visual language for understanding the context of data.

1.See the Pew Research Center Internet & Technology’s Americans’ Views on Open Government Data, available at http://www.pewinternet.org/2015/04/21/open-government-data.

2.See the Los Angeles GeoHub at https://geohub.lacity.org for more information.

3.See the UNSD SDGs website at https://unstats.un.org/sdgs/indicators/indicators-list for more information.

4.See Arcade Expressions and You, available at https://arcgis-content.maps.arcgis.com/apps/PublicGallery/index.appid=8951b538362b492cadadf7ede1b85c21.

5.This workbook is available at https://www.esri.com/library/books/the-language-of-spatial-analysis.pdf.

6.See Esri documentation at https://pro.arcgis.com/en/pro-app/tool-reference/spatial-statistics/h-how-hot-spot-analysis-getis-ord-gi-spatial-stati.htm and https://pro.arcgis.com/en/pro-app/tool-reference/spatial-statistics/hot-spot-analysis.htm for more information.

7.See the 2017 TechRepublic^® article Understanding the Differences Between AI, Machine Learning, and Deep Learning by Hope Reese, available at https://www.techrepublic.com/article/understanding-the-differences-between-ai-machine-learning-and-deep-learning.

8.McKinsey (https://www.mckinsey.com) is a global consulting company that conducts qualitative and quantitative analysis to evaluate management decisions across the public and private sectors.

9.See the 2018 Microsoft article Microsoft and Esri Launch Geospatial AI on Azure by Joseph Sirosh, available at https://azure.microsoft.com/en-us/blog/microsoft-and-esri-launch-geospatial-ai-on-azure.

10.See Microsoft and Esri Launch.

11.See the Esri Newsroom article “A new business intelligence emerges: Geo.AI” by Mansour Raad, available at https://www.esri.com/about/newsroom/publications/wherenext/new-business-intelligence-emerges-geo-ai.

12.See the 2019 Esri article Think Tank: Blockchain Evolves into Geoblockchain, available at https://www.esri.com/about/newsroom/publications/wherenext/geoblockchain-think-tank.

13.See https://en.wikipedia.org/wiki/Blockchain.

14.See the CNBC article Blockchain Could Be a $7 Billion Market and a Major Book to Amazon, Microsoft, Bank of America Says, available at https://www.cnbc.com/amp/2018/10/02/blockchain-could-be-a-major-boost-to-amazon-microsoft-analyst-says--.html.

15.See Blockchain Could Be a $7 Billion Market.

16.See the 2018 UNDP article Using Blockchain to Make Land Registry More Reliable in India by Alexandru Oprunenco and Chami Akmeemana, available at http://www.undp.org/content/undp/en/home/blog/2018/Using-blockchain-to-make-land-registry-more-reliable-in-India.html.

Goal 1. End poverty in all its forms everywhere
Goals and targets (from the 2030 Agenda for Sustainable Development)	Indicators
1.1 By 2030, eradicate extreme poverty for all people everywhere, currently measured as people living on less than $1.25 a day	1.1.1 Proportion of population below the international poverty line, by sex, age, employment status, and geographical location (urban/rural)
1.2 By 2030, reduce at least by half the proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions	1.2.1 Proportion of population living below the national poverty line, by sex and age 1.2.2 Proportion of men, women and children of all ages living in poverty in all its dimensions according to national definitions
1.3 Implement nationally appropriate social protection systems and measures for all, including floors, and by 2030 achieve substantial coverage of the poor and the vulnerable	1.3.1 Proportion of population covered by social protection floors/systems, by sex, distinguishing children, unemployed persons, older persons, persons with disabilities, pregnant women, newborns, work-injury victims and the poor and the vulnerable
1.4 By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance	1.4.1 Proportion of population living in households with access to basic services 1.4.2 Proportion of total adult population with secure tenure rights to land (a) with legally recognized documentation, and (b) who perceive their rights to land as secure, by sex and type of tenure
1.5 By 2030, build the resilience of the poor and those in vulnerable situations and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters	1.5.1 Number of deaths, missing persons and directly affected persons attributed to disasters per 100,000 population 1.5.2 Direct economic loss attributed to disasters in relation to global gross domestic product (GDP) 1.5.3 Number of countries that adopt and implement national disaster risk reduction strategies in line with the Sendai Framework for Disaster Risk Reduction 2015–2030 1.5.4 Proportion of local governments that adopt and implement local disaster risk reduction strategies in line with national disaster risk reduction strategies