Chapter 11

Serving in the Public Sector or Academia

In This Chapter

Learning about life in academia

Looking at public sector big data responsibilities

Identifying big data roles that impact healthcare

Not all the jobs in big data are concerned with driving profits or revenue. Many roles in big data out there are in the public sector or university settings working toward public safety, medical discovery, and the advancement of science. Instead of looking for the best pricing combination or executing a online shopping basket analysis, you may find yourself wanting to uncover future signs of cancer, predict where the next disease may strike, or run simulations on clinical trials to find a cure for a disease.

This chapter covers the role big data plays in different areas of the government, how higher education leverages big data, and how you can explore the job possibilities.

The Role of Academia in Advancing Big Data

Higher education plays a very important role in the big data ecosystem. From training new talent to pushing the envelope of innovation, there are many exciting areas available for people interested in big data careers.

Teaching at the college level

The foundation of academia is teaching. The role of the university is critical for teaching the next crop of mathematicians, scientists, and business analysts. If you’re considering a role as a professor, you’ll find big data to be a very rewarding and challenging field.

Here’s the typical career path of a college professor after earning a PhD:

• Assistant professor: The role of assistant professor is for newly minted PhDs. It’s an entry-level job at a university or college that puts the professor on a tenure track. If the professor doesn’t rise to the rank of associate professor within eight years, he’s usually dismissed.

  Tenure is a status that is granted to select professors guaranteeing them employment after they complete a probationary period. Professors are tenured to ensure that they won’t lose their jobs because of political infighting or other nonteaching issues. Tenure provides them the academic freedom to teach in their own way without fear of repercussions.

• Associate professor: The role of associate professor is the next step up toward tenure from assistant professor. Associate professors are generally experienced academic researchers. Not all associate professors are on a tenure track. Those who aren’t may remain at this level and won’t go on to gain tenure.
• Full professor: Professors who have risen through the ranks from assistant professor to associate professor are promoted to full professor. Full professors are very senior members of the faculty and have tenured positions.
• Endowed chair: Endowed chairs, sometimes called distinguished professorships, are honorary positions linked to an endowment that funds the person’s salary. You know these individuals because they have titles like one of my colleagues at the University of Virginia, Thomas Bateman, Bank of America Eminent Scholar, Professor of Commerce. That’s a mouthful!

Though the academic path is straightforward, it isn’t easy or automatic. In most cases, when an assistant or associate professor is being considered for the next level, everything is on the line. If tenure isn’t granted, he’s often dismissed from the university. Competition is tight, pressure to publish is high, and strong performance in the classroom counts. Remember those surveys you used to fill out at the end of a class? Those are used for the professor’s career progress.

What about your buddy at work who also teaches a class at the local university? You can be that person, too. There is a class of educators at the university level who are not on the formal professor track. They might be called lecturers or adjunct professors. These titles are reserved for part-time faculty members who are at the university for teaching purposes and usually have other jobs. They don’t usually have the research or service requirements that other faculty members have — they’re focused on classroom teaching. All schools, from the largest universities to the smallest community colleges, employ part-time faculty to help meet classroom demand or bring in professionals with specialized knowledge to add value to their programs.

Conducting research

Universities are centers for research. When you think of research, you may imagine lab coats, bubbling beakers, and Bunsen burners. But much of the research that is conducted today is focused on data analysis. There are a variety of research jobs in universities — from research assistants all the way to post-doctorate researchers. These roles are often funded through grants from the National Science Foundation, the Department of Energy, the Department of Defense, the National Institutes of Health, and many other agencies in conjunction with university research. Some of the positions may last only as long as the funding lasts.

Nonprofit Industry Organizations

Nonprofit organizations often serve an entire industry as a whole. They’re funded by member companies to drive the following:

• Big data research: Some groups fund projects to explore new areas in big data. These funds can be used to conduct research new software in big data or ways to apply the technology.
• Technology standards: As industries develop, there is a growing need for people to agree on a common set of ways to exchange information, define terms, and integrate systems. It’s like grammar for a language. When a group can agree on the rules of how words should work, it makes it much easier to communicate.
• Awareness: These groups conduct events, conferences, or publications to promote general awareness of big data technology.

Companies and organizations often come together to build information and data systems that all member firms may benefit from, even though they’re competitors in some cases. For example, The Data Warehouse Institute (http://tdwi.org) is made up of members from competing firms. However, they collaborate to promote the industry as a whole.

Another example is the National Association of Homebuilders (NAHB), an organization that has been around for more than 70 years and has 140,000 member firms. Homebuilding represents a huge part of the U.S. gross domestic product (GDP) when you factor in all the associated trades and industries. Data, insights, and research from the NAHB impacts financial markets, public policy, building and safety standards, and a number of other key issues. Figure 11-1 shows a heat map representing the leading building markets in the United States to illustrate the new construction housing recovery. Heat maps show concentrations, or heat, of groups of data by a concentration of a particular color.

Courtesy of the National Association of Home Builders

Figure 11-1: An NAHB housing heat map.

Organizations within the Public Sector

Public sector jobs are typically segregated into civilian and defense. Their customers are the agencies or organizations they’re tasked to serve. These groups can be local, state, or federal. Public sector jobs are focused on supporting infrastructure, services, public safety, and regulatory needs. For example, in public safety, you may analyze crime data to predict where to apply funding for a police force, or you may analyze disease data to help provide input for policymakers and lawmakers. The insights from big data can have a huge impact on the government.

McKinsey & Company, a management consulting firm, has identified three major areas for the uses of big data in public sector administration:

• Operational efficiency: Gaining savings by identifying better ways to get things done.
• Reduction and prevention of fraud: Helping to spot abuse and fraud by identifying patterns in claims, whether medical benefits or tax returns.
• Increased tax revenue collection: Finding where to apply regulations to help increase tax revenues by identifying tax gaps in businesses and individuals.

Big data can also be used to predict potential crime networks, terror connections, and other crime risks. Big data analysis can be used to spot and predict health and disease information that can be used by policymakers who are appropriating funding for medical research.

Another useful way that the federal government can use big data is to find ways to reduce taxes and to spend tax dollars more efficiently.

Civilian organizations

Civilian organizations are local, state, and federal government groups that are not associated with defense and intelligence. These include law enforcement groups like the local police and the Federal Bureau of Investigation (FBI), which is a part of the Department of Justice.

Life in these groups differs vastly from one agency to the next, but they do share a lot of the same characteristics:

• The application processes are tightly regulated. It isn’t as easy as calling your buddy who does the hiring. Public organizations generally have many applicants and need to follow a fair and regulated process.
• The pay scales will likely be less than in the private sector. Raises are often at the mercy of the local economy, state governor, or organizational head.
• Most of these groups have appealing retirement plans for those who are able to participate. This long-term security can be worth a bit of lower pay during your career.

Somebody once said that steady plodding brings prosperity. So if you’re inclined to leverage your skills for the benefit of society, the public sector could be a great fit!

Defense and intelligence

Some government agencies have been leveraging big data to run analytics to identify potential threats to the United States. These threats include such things as spying from foreign countries, attacks to our financial institutions, and cyber-terrorism. The task of sifting through unstructured data like voice files (which could include recorded phones calls or voicemails) and emails in search of patterns to identify future threats is a classic case of using big data.

Consider an example of government intelligence agencies that need to isolate events related to suspected conversations related to terrorist activities. These events can occur via emails, text, voice, and other media. The first step would be to isolate noise from the relevant content, creating a group of interest. With big data, agencies can

• Identify potential terrorist threats in near real-time by running actionable advanced analytics on all available data.
• Aggregate critical data from disparate sources in real-time including documents, email, audio and video files, social media posts, and blogs.
• Perform behavioral analysis to isolate risk and threats, and understand behavior within and among groups.
• Build targeted identification campaigns for specific suspect segments.
• Perform suspect behavior analysis to predict potential threats and anticipate suspects’ behavior and movements.
• Communicate real-time decisions and desirable actions effectively among all impacted persons and emergency personnel.

Given the technical and data requirements, you can see that government agencies have a high demand for big data analysts. People who enter a three-letter agency like the FBI, Central Intelligence Agency (CIA), National Security Agency (NSA), and Defense Intelligence Agency (DIA), enter at a rank structure somewhat similar to the military. There are job classifications that determine the level of responsibility and associated pay grades.

You hear these letters in the news, books, and movies, but what do they mean and what do they do? Here’s a quick rundown:

• CIA: The CIA’s main purpose is to gather information on foreign governments, firms, or people, and provide that information to U.S. policymakers. Big data jobs can be used to analyze information to help predict where the next threat to the United States may come from.
• DIA: The DIA is similar to the CIA, but it focuses on foreign military capabilities. Big data jobs are focused on data analysis of information related to military operations.
• FBI: The FBI is the U.S. Department of Justice’s group to investigate crimes on behalf of the federal government. This includes organized crime, terror, and other federal crimes.
• NSA: The NSA’s main goal is to monitor global communication for intelligence gathering and to protect the U.S. communication capability.

Figure 11-2 shows a posting taken from www.cia.gov for a data scientist within the agency. As you can see from this posting, the salary range is quite wide and depends totally on your experience level. Much of the work is similar to a corporate IT job within the commercial sector. The main difference is that the objective of the analysis involves national defense instead of assisting business units in increasing profitability.

Figure 11-2: A posting for a data scientist job at the CIA.

Healthcare and Medical Research

McKinsey & Company is a prestigious consulting firm that works with leaders in business and government around the world. It made a huge impact in the world of big data with some very big claims in its 2011 report, Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey projected the impact of big data on healthcare in the United States could top $300 billion per year, which is more than double the total annual health spending of Spain.

Using information to improve healthcare is in the infancy stages today, which should bode well for the future of big data job growth. There are three factors to support this:

• The increase of digital usage for patients like electronic medical records.
• The lack of integration that exists today across healthcare providers, payers, governments, and drug companies.
• The amount of data that will result from the increase in usage of complex data systems from research to treatment.

Healthcare companies need to derive insights from the data they collect, but this is complicated by a lack of system integration. This lack of integration is only an opportunity for growth in jobs, responsibility, and innovation. This means that big data people entering the workforce today will be part of a huge movement to bring massive advancement across this complex network. There is great room for innovation when gaps like this exist.

You may wonder what this means for big data? This leads us to the final factor that promises huge job potential for big data workers in the near future and long term. With millions of people using digital devices within their healthcare network, there will be an enormous glut of data from which we can pull untold insights. I say “untold” here, because we just don’t fully understand what will be discovered. That is the big promise of big data!