GIS and the 2020 Census: Modernizing Official Statistics

As mentioned in chapter 4, up-to-date imagery has proven to be a critical source of information to support NSOs as they plan and execute their national census efforts. This chapter will explore in detail the key use cases of imagery for national census programs. Imagery, if used practically, can save incalculable work time and allow the NSO to focus on areas of significant change or growth. Imagery and remotely sensed data can also be used to cover dangerous or inaccessible areas, saving countless hours of fieldwork if done properly. Characteristics of population that can be analyzed using satellite imagery include counts of dwellings, measurements of settlement areas, measurements of areas that have had significant land-use change (e.g., agriculture and forestry), and residential increases.

In this chapter, our primary focus will be on satellite images, but aerial imagery should also be considered. Satellite images are obtained from space-based systems and are collected by governments and businesses around the world. Aerial photography traditionally was captured using cameras on low-flying planes, but today can also be obtained by using UAVs or drones.

Imagery has key applications that benefit the national census program, including the following:

•Imagery basemaps—Provide highly updated context for planners and enumerators. Assists in locating census targets and delineating appropriate EAs.

•Change detection—Comparing imagery from different times lets enumeration planners focus their mapping and feature collection efforts based on areas that have had the most change since the last census.

•Updating basemap and feature layers—Use recent (high-resolution) imagery to ensure that roads, utilities, buildings, and other infrastructure layers are up-to-date for accurate canvassing strategies.

•Agriculture and natural resource assessment—Use imagery to map the extent of specific farms, agricultural areas, and other natural resources for inventory purposes.

•Visualization—Imagery provides an unadulterated view of the land whereas maps can have misrepresentations and omissions. Tools to visualize imagery empower apps on mobile devices used for enumeration.

Imagery can be used at all phases of the statistical business process model. This table provides a general usage model for imagery during each phase of the enumeration process.

The various techniques listed in figure 5.3 are used to create the numerous information products that support the statistical business process model. Esri provides software tools to support all these techniques. These tools are a seamless part of the ArcGIS platform. The following provides a short description of each technique and the relevance of the resulting information product to the enumeration processes:

•Ortho mapping—This process is used to make imagery as geospatially accurate as a map. It is the most critical process used in the creation of any type of imagery product for the census effort, particularly for making basemaps and performing change analysis. The ArcGIS platform provides ortho-mapping capabilities for all types of imagery, including satellite, aerial, and drones. Because the ortho-mapping process requires some specialized skills (depending on the source), many organizations require that the imagery vendor provide imagery already orthorectified and map-accurate.

•Imagery management—Operating a nationwide imagery program capable of supporting a census effort requires a robust image management capability because imagery files can readily exceed multiple terabytes of storage. Two key functions of imagery management include cataloging and serving of imagery content. Cataloging is used to ensure the efficient discovery of all imagery holdings for both manual and automated processes. Image serving ensures the efficient delivery of imagery content to end-user applications typically via web services. ArcGIS^® Image Server is ideal for managing and serving imagery and readily scales to meet any amount of imagery data.

•Image analysis—This manual process is performed by an image-processing specialist to extract information from imagery. It is used to update GIS feature layers such as roads, building centers, and other infrastructure features that are key to an EA. Workflows typically include some form of visual change detection so that existing feature layers can be updated from the most recent image covering the interest area. ArcGIS^® Pro provides a variety of tools to aid with the manual analysis and extraction features from the imagery. This functionality even includes a new stereo viewer for working in 3D. While manual extraction can be time consuming, it is often much more accurate than using automated ways to extract features from imagery, especially for constructed features in urbanized areas.

•Image analytics—This process includes highly automated methods of extracting information from imagery. Proven image analytical capabilities include automatically mapping land cover, identifying areas of greatest change, and evaluating agricultural health. With high-resolution elevation data, ArcGIS now includes analytics for the automated extraction of building footprints. While there are currently no “easy buttons” when it comes to feature extraction, new AI techniques are showing promise and may become available by 2020. Image analytics will continue to improve the ability to rapidly extract the key geospatial information needed to perform a modern national census.

•Imagery apps—These apps include imagery-specific widgets that enable enumeration and apps on mobile devices that take advantage of imagery. These JavaScript widgets contain functionality such as automated contrast and brightness tools, image sharpening, swipe comparisons, and image measurements. Web AppBuilder for ArcGIS^® can be used to build enumeration apps.

•GIS analytics—With ArcGIS, all imagery-derived products can be used in advanced GIS statistical analysis because accessing and processing images are integrated into the ArcGIS platform.

Different sources of imagery have different characteristics suitable for NSOs. Because the acquisition of imagery can be expensive and difficult to maintain, careful consideration must be given before deciding the proper mix of imagery. Key characteristics that must be considered include image resolution, accuracy, spectral resolution, spectral fidelity, and obliquity. Elevation data is also important and should also be factored in the decision.

•Resolution—Image resolution is undoubtedly the most critical characteristic of imagery for national census programs. It provides enumerators the ability to visually identify all the various structures that may potentially house people or businesses and the infrastructure that supports them. In general terms, urban areas need a resolution of at least 30 cm while rural areas should have a minimum of 50 cm. Once resolution is worse than 50 cm, it becomes difficult to properly distinguish the key geospatial features needed to help guide enumeration. The two exceptions for this premise are imagery that is used to identify generalized areas of change and basic land cover/use. In both cases, where specific structures do not have to be identified, a medium resolution from 1 m to 30 m may be adequate.

•Accuracy—All imagery must be orthorectified and should be accurate to within one-half pixel of resolution. For instance, 30-cm imagery should be accurate to 15 cm. The ability to quickly collect accurate imagery is improving, so updating GIS layers from inaccurate imagery will quickly become obsolete and unusable.

•Spectral resolution—The concept refers to the number of bands each image has. True color images contain three spectral bands: red, blue, and green. Many aerial collection systems add a fourth near-infrared (NIR) band to help with vegetation analysis. In practice, the more spectral bands, the better automated feature extraction (analytics) can work. For enumeration work, three-band true-color imagery will work, but the fourth NIR band is a good choice if not overpriced. (Aerial collectors will typically acquire the NIR band regardless, so strongly negotiating a reduced price for this band is recommended.) For extracting land cover and performing change detection, more bands (even if they have less resolution) is preferred.

•Spectral fidelity—The concept refers to the color quality of the imagery. Sources with good spectral fidelity are better for building seamless mosaics for basemaps and automated feature extraction. Remember that atmospheric effects can have a huge impact on spectral fidelity, so it is often difficult to build quality basemaps from satellite imagery.

•Obliquity—Most images for basemaps are taken as nearly vertical as possible (in other words, the camera is pointing straight down). However, the trend is growing toward oblique imagery taken from the side. For enumeration purposes, this method allows users to see things not apparent in vertical imagery, such as building sides for counting the number of floors in an apartment building. The downside to obliquity is that it does not make good basemaps and cannot be used for analytics. Users should also be aware that high-resolution satellite imagery is also often oblique, and because collection angles vary from image to image, this obliquity is not uniform and results in significant distortions in urban areas.

•Elevation data—High-resolution elevation data is a key source of geospatial information. Once difficult to collect, new lidar sensors and elevation extraction techniques have made this source more prevalent. For enumeration purposes, it is currently most useful for the automated extraction of building footprints.

The following is a portfolio of imagery programs provided by Esri or its content partners. Each program listed has been optimized to work with the ArcGIS platform with minimal implementation efforts.

•Esri imagery basemap—The Esri imagery basemap web service has been compiled by Esri from different sources of satellite and aerial imagery. It provides moderate- to high-resolution imagery data covering the entire globe. In many parts of the world, including most of Africa and the Middle East, the service has recently been updated with DigitalGlobe^® satellite imagery collected from 2016 to 2017 with an average resolution of 50 cm. Efforts have been made to color balance the imagery so the basemap is as seamless as possible at all resolutions. The service is free to all ArcGIS users. If web services are not suitable for the application, the imagery can also be acquired on an Esri solution known as the data appliance. The imagery itself is served as a tile cache, meaning that it is not suitable for analytics. But Esri does provide relevant metadata, such as source and acquisition dates, for all the imagery in the archive. The imagery and metadata can be accessed via this link: https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer

•Woolpert national imagery map—Woolpert is an Esri business partner that specializes in the collection of high-resolution aerial imagery and elevation (lidar) data. It has recently implemented a collection program to cover key countries throughout Africa and the Middle East. Its products include four-band 15 cm imagery over urban areas and four-band 25 cm imagery over rural areas. Its imagery is of high-color fidelity, so it is ideal for seamless imagery basemaps and can be used in automated extraction processes. Woolpert’s collection activities have recently begun, so all its imagery will be extremely up-to-date for the 2020 census time frame.

•MDA/DigitalGlobe Imagery + Analytics (I+A)—I+A is a joint program of Esri and DigitalGlobe to provide high-resolution satellite imagery for basemapping and analytics. I+A expands DigitalGlobe’s Geospatial Big Data Platform (GBDX) program with image-processing functionality and ready access to its imagery archive via an ArcGIS image server interface. Access to I+A imagery is provided on an annual subscription basis and is updated on a fragmentary basis depending on cloud cover and the collection rate of its satellites. Its absolute best resolution is 30 cm but actual images can range from 30 to 60 cm depending on the look angle of the satellite. Imagery fidelity varies depending on atmospheric effects and different collection dates and time.

•Copernicus—Copernicus is the European Union’s Earth Observation Program for which the Sentinel satellites are developed to meet its specific needs. Sentinel-2 provides high-resolution imagery for land services, including imagery of vegetation, soil and water cover, inland waterways, and coastal areas. Esri enhanced ArcGIS technology to simplify the use of free global imagery from Sentinel-2. ArcGIS supports the visualization, interpretation, and analysis of Sentinel imagery, which is valuable in applications of agriculture statistics and forestry statistics as well as land and environmental monitoring.

•Light detection and ranging (lidar)—This relatively new remote-sensing technology allows us to collect very dense point samples of features in 3D. Lidar technology has evolved to become a common source of geographic data in GIS. Lidar data is typically stored in LAS files. Each lidar point can have additional attributes such as intensity, class code, and RGB color values, which can be used in ArcGIS. ArcGIS reads LAS files natively, providing immediate access to lidar data without the need for data conversion or import. LAS attributes can be used to filter content and symbolize points in 2D and 3D. Lidar can be used for feature extraction, validating existing GIS data, and measuring heights and distances between points.

•Landsat—The US Department of the Interior has made Landsat information available for the whole world for free—over eight terabytes of multispectral imagery collected over the last thirty years that describes how the planet has changed. Esri has developed a website called Change Matters (http://changematters.esri.com/compare) that provides fast access to this huge reservoir of data. It is made accessible through image services in ArcGIS. This is not just an image in a browser; it allows you to perform a query about a particular geography, and the server will process it into a continuous mosaic that can be analyzed dynamically. The earth is constantly changing; this new collection allows users to measure change over time and help them understand the changes using a web browser.

Many other imagery providers are available. Depending on your needs and geographic area, the options will vary. Often, the NMA maintains a national imagery program for the country. If the NMA does not have imagery available, the NSO may need to consider other ways to compile image services capable of supporting its imagery needs. This area may be considered for contract or managed services. Specialists familiar with all types of satellite, aerial, and drone imagery can build seamless tile cache basemaps or true imagery services for analytics. Common sources of imagery for managed services include aerial and satellite imagery already owned or being collected by the NSO, Landsat satellite imagery, and Sentinel-2 satellite imagery from the European Space Agency (ESA). By outsourcing the compilation and management of the imagery service to experts, valuable time and critical personnel in the statistical organization are freed for the actual tasks of building the imagery-derived information products needed to support a national census.

An NSO conducting a national census using imagery may need a dedicated platform that combines imagery with GIS. Starting with ArcGIS version 10.5, a new image server has been added to complement ArcGIS Enterprise. This image server allows organizations to separately scale their imagery program to best meet the data loads and services they want to provide. A new image catalog app facilitates imagery cataloging and management. Recent additions for ArcGIS Desktop and ArcGIS Pro include the ortho-mapping workflow, the extraction of 3D features from stereo imagery, and automated feature extraction and change detection. As drones become more prevalent, Drone2Map^® for ArcGIS provides a robust processing capability for these emerging sources of overhead imagery. All these capabilities can run on in-house systems or in the cloud. Organizations can start with a basic desktop single-server approach and then expand their capabilities as their imagery needs grow. ArcGIS now provides all the imagery capabilities needed by NSOs for national census and inventory programs. Furthermore, through Esri content providers, numerous sources of imagery content are available that meet the needs of the national program.

In summary, the many advantages of satellite remotely sensed data and aerial imagery include the following:

•Imagery can be used as a validation in field data collection and verification.

•Up-to-date coverage can be obtained at relatively low costs with lower spatial resolution.

•Imagery can be applied to multiple surveys and census, and once acquired can be applied to other areas and applications.

•Aerial imagery typically shows a large amount of detail and thus allows for visual analysis.

Satellite earth observations (EO), in addition to their use for the census, can support measuring and monitoring the global indicator framework for the SDGs. In fact, the Group on Earth Observations (GEO) has an initiative⁵ to organize and realize the potential of EOs and geospatial data to advance the 2030 agenda and enable societal benefits through achievement of the SDGs. Additional information on GIS and the SDGs will be provided in chapter 9.

Recommendations

For NSOs wanting an enterprise imagery management system, the following are recommended guidelines for system configuration and imagery content:

•System configuration—NSOs can start small with their imagery programs but must be smart enough in their configuration strategy to allow growth. The first step in modernizing an imagery approach is to move away from separate imagery files to a web services approach supported by the new ArcGIS Image Server. A file management system will not scale. On the other hand, Image Server can work with millions of image files to automatically find the correct image pixels needed for the basemap being visualized or the analytical process being run. A baseline configuration includes ArcGIS Desktop with ArcGIS Pro, the new Image Analyst extension for ArcGIS Pro, ArcGIS Enterprise, and ArcGIS Image Server. This system can reside on premises, in the cloud, or be a hybrid of both.

•Imagery content—For NSOs, it is recommended that high-resolution four-band aerial imagery be used for basemapping and manual feature extraction such as the imagery being provided by the Woolpert National Imagery Map Program. For change detection, land cover extraction, agricultural assessments, and other analytics, Esri recommends Sentinel-2 satellite imagery from ESA’s Copernicus program. A possible alternative to both these sources is DigitalGlobe satellite imagery, especially if the new competitive pricing of Access + Analytics can beat out the aerial sources. For smaller high-density areas and areas with rapid change, NSOs should consider implementing a drone program.

See the online ancillaries at esri.com/Census2020 for a list of imagery providers.

Case study: Portugal

To prepare for the 2021 census, Instituto Nacional de Estatística Portugal (INE) initiated a work plan for creating the geospatial components of Portugal’s census operation. The plan is built on the agency’s location-based strategies established two decades previously, when INE first included GIS in its 2001 census. The agency realized that any data that links people, business, and the economy to a particular place offers a more complete understanding of social and economic issues. From then on, INE has been committed to making geospatial data an integral part of its work.

Today, geospatial data is present in most phases of INE’s statistical production processes. The country’s goal is that through the census, every statistical unit, person, household, dwelling, building, and business register will be geocoded.

In 2006, INE developed a spatial data infrastructure (SDI) that adds a geospatial component to all phases of statistical production. The SDI is more than a sequence of mapping operations and census data dissemination. It is a continuous digital transformation program that makes it possible for INE to evolve its operations, meet changing demands for information, and stay current with society’s technology expectations.

To understand Portugal’s census program is to understand the European Union’s (EU) census program. INE follows recommendations and standards set by the EU’s statistical office, Eurostat. Agencies that follow the EU’s General Guidelines of Statistical Activity have highly credible and authoritative data, thereby giving researchers confidence in the data for analysis. Furthermore, EU standards make it possible for Portugal to compare its statistics with those of other member states. Portuguese economists, for instance, can compare unemployment, housing, and other economic indicators with those of France.

When it comes to geo-enabling statistical production in Europe, two EU initiatives set the course. One is Eurostat’s GEOSTAT, which defines the procedures for processing official statistics, including using generic statistical models that integrate geospatial data. The other is the European spatial data infrastructure (INSPIRE), a directive that state members follow to build the European network for sharing geospatial information. To make European data compatible, INSPIRE defined data standards for thirty-four spatial data themes such as transportation, utilities, and population. Each country builds these into their data models.

In Portugal, INE works with the strategic body responsible for implementing the INSPIRE directive throughout the country. INE mainly works on developing INSPIRE’s metadata and services. It has also built data themes for Portugal that are similar to INSPIRE’s data themes.

Although EU initiatives prescribe the approach for developing geospatial data, it is Portugal’s Census 2021 that drives the nation’s integration of statistical and geospatial information. For instance, Portugal is updating procedures and work processes that respond more efficiently to the challenges of organizing fieldwork and data processing. INE also developed social survey processes for data sampling. The agency uses GIS to manage procedures and automate work processes that simplify tasks.

INE’s geoinformation group manages the Buildings Geographic Database (BGE), which contains point-based data for all residential building units. INE uses this georeferenced data to generate tailor-made statistical products at different scales, such as a 1 km² grid map that includes a range of census attributes. GIS is used to check the quality of three data types. The EAs are blocks having three-level structure (sections, subsections, and localities), which are integrated with official administrative boundaries. The Road Segments Network is street-line coverage at national and local levels. Data is edited with geometric and alphanumeric data submitted by municipalities. This is used for the delineation of the agency’s geographic information referencing base (BGRI). Finally, GIS is used to check the aforementioned BGE building locations data. The agency’s quality-control process increases the accuracy and consistency of every building’s x,y coordinates and address.

All of Portugal’s municipalities share their building permit information and completed-construction work data with INE via a web GIS platform. To check the data, the geoinformation group follows SDI quality-control procedures, which are predominantly GIS-based quality routines. To identify topological and attribute errors, the group uses ArcGIS software to check the data against orthoimagery and boundary data provided by Portugal’s cadastral agency. They then make edits accordingly.

Figure 5.18. BGE buildings location data.

The entire GIS is a hybrid system that includes ArcGIS and Oracle^® as well as some open-source software. In addition, INE developed web mapping applications to help with data sampling, collection, and dissemination procedures. For the 2021 census data collection, INE has been developing web mapping applications that make relevant spatial data available to all enumerators and data collection managers.

Portugal is relying on GIS to help it efficiently execute the Portuguese 2021 Census and ensure the availability of quality information. However, the country’s plans for its GIS extend years beyond the upcoming census. INE will use GIS to transition to a future census that will be based on administrative registers. In doing so, INE looks forward to decreasing the statistical burden on citizens, improving the frequency of outputs, and reducing collection costs associated with census operations.

Figure 5.19. GeoINQ GIS web app.

Phase	Application	Technique
Planning Pre-Enumeration	•Create accurate enumeration basemaps •Identify areas of change from last census •Update infrastructure feature layers •Extract building footprints •Update the extents of EAs	•Ortho mapping •Image analytics •Image analysis •Image analytics •Image analysis
Enumeration	•Imagery basemaps on mobile devices •Imagery visualization tools for devices •Agricultural analysis and assessments •Land resources analysis and assessments	•Imagery management •Imagery apps and widgets •Image analytics •Image analytics
Dissemination Post-Enumeration	•Imagery-based story maps and reports •Updated basemaps •Derived agricultural products (e.g., yields) •Derived land resource products	•Imagery management •Imagery management •GIS analytics •GIS analytics