Nearly a year ago during one of many discussions regarding the impact of big data at companies struggling to realize its full potential, the topic of data visualization and visual communication became the centerpiece. While many companies had found proficiency in working with big data or even created data science and analytics initiatives, enabling people to work properly with data visualization tools was falling far short thus creating a roadblock for full realization of big data’s potential.
This is the visual imperative, ultimately driven by every company’s need to enable its people to analyze and communicate effectively in a data-centric world. Lindy and I started thinking about ways our research could help companies take something that started as a uniquely individual passionate and grass roots movement and turn it into an enterprise-wide program.
From there, Lindy’s passion for the topic, depth of experience, extensive research, and practical application grew. The chapters in this book represent this body of research and ongoing mission, and as I consider the importance and potential value for readers, four pressing drivers rise to the top.
Imperative 1. Big data
Everyone might be a little tired of hearing about big data but it’s a reality that there is so much more data for companies to leverage today, and it is increasing exponentially and coming at even faster speeds. We simply cannot handle it all, crunch it all or make sense of it all with the techniques we have used in the past. Business demands are driving technical innovation in computing power, storage density, and bigger faster distributed architectures, and the fact is that we are always trying to play catch up with the amount of data that is out there.
What we are experiencing now is similar to the industry-wide shift we saw 15 years ago when saying business intelligence was not just about the data warehouse. Then—and now—we go through the cycle, first trying to figure out how to manage all of the new data (an order of magnitude larger than what’s currently in use) and then figuring out how to leverage that data for insights. Data warehousing gave birth to BI and performance management in this same way. The technologies and buzzwords have evolved, but going through a similar cycle now with big data (Hadoop) having arrived around 2008, followed by a growing need to leverage big data differently via advanced analytics (like the current machine learning and cognitive computing movement) and advanced visualizations and user engagement. While data is the key ingredient, computations and the ability to make business insights and decisions on the data is always the goal. It is not simply about having big data. It is what you do with it.
The cycle of technological innovation drives advances in both managing and analyzing the data. In recent years, the maturing and mainstream adoption of Hadoop has begun to stabilize the ability to acquire and manage big data sets as an extension of enterprise information management. Now we need to know how to work with, view, and understand all of it. In the BI era, visualizations had a well-defined role for communicating information reports, analyzing data dimensionally, and displaying goal-oriented dashboards. However, traditional data visualizations fall short in the big data era where the need to explore new data sets and discover insights is equally coupled with the ability to communicate all the patterns and relationships within the data in new ways.
What this means for data visualization is that there is no way to comprehensively look at, interact with, represent, and collaborate over that much data except to visualize it effectively. This is the imperative that will only increase with more and more data. New data sets are changing the landscape, and we are still only scratching the surface of what is possible. Internet of Things and machine data will take that an order of magnitude higher again. If you want to communicate with people or with machines, the way to do that will be through visual communication. If you are going to work with big data—today and in the future—you have to be competent at visualization, and that competency must be being taught, evolved, and monitored within companies today.
Imperative 2: The need to engage through design and user experience
The far-reaching impact of the technology revolution of 2007 with the launch of the Apple iPhone is not to be underestimated. Across every industry, Apple has had a profound influence through the psychological effect of how consumers expect technology to interact with them. People now expect good design as part of their visual communication and interactivity with information. In obsessing over simple, intuitive design, Apple sets a new standard for visual communication. It was necessary to design a completely original visualization experience because when converting from the real estate of a computer monitor to a 3.5-inch screen, smart choices must be made to effectively communicate visually and create intuitive interaction with the device. Not that Apple has always done this perfectly, but it has always focused on aesthetics and the user experience.
In fact, Apple went through a phase of “iCandy” with its photo-realism aspect. But then it adopted a whole new level of visual effectiveness that went away from what looks especially pretty and realistic, and went to flattened minimalism and straightforward iconography. The result may not be quite as pretty, but it is significantly more effective because it increases visual efficiency.
So you can see that whether it is massive amounts of data to work with or this new era of design we live in, the stage is set for what I think people have come to expect and perhaps even set the bar for acceptance criteria. People simply cannot go from the optimal visual designs and user experiences of iOS (or Android) and then be content with a laptop or tablet with a clunky interface. An earlier example of this was what I called the “Google Effect” when Google conditioned users to expect 100,000+ search results in 0.4 s. It was difficult that in my personal life I had ease of use and highly iterative and agile query and exploration or discovery ability, but at work I had to wait five minutes (or more!) for a standard sales report. Data visualization has become the new market to supplant where BI has not been able to meet user expectations. Apple did the same for visualization. The “Apple Effect” changed the expectation for aesthetics and interactivity. It influenced the entire product world across industries, as consumers now have exceptionally high standards for visual experience.
The important point here is that true visual efficiency is the expectation: stripping away everything so you are accurately facilitating discovery, enabling analytics with visualization, communicating an insight, or telling a data story and not being distracted because something is pretty or an artistic expression of data.
Imperative 3: Visual communication
With big data being managed and the latest generation of powerful data visualization tools on the market for companies to leverage, the real challenge then moves away from a technology issue to people and processes issue. Just because I exchanged your hammer for a nail gun and your hand saw for a cordless circular saw, does that mean you can build a skyscraper? When communicating data to any audience, proper data visualization and representation still does not guarantee accurate or effective communication. One must also consider the mode of the reader/audience and the language of visual communication for different kinds of conversations—sharing, understanding, exploring, persuading, and storytelling.
When creating a visualization for a lot of data, my task is not completely finished until I structure my presentation of the data to solicit a response or engage the viewer. I need to think through the flow of the visual experience and what the user will be seeking from the data. User experience design and visual communication go far beyond whether a bar chart or pie is the correct way to represent data. Proficiency is derived not only from a formal skillset but also from the narrative that we hear from the inner voice in our heads when we develop data visualization stories. The techniques come from a long history of humans telling stories and connecting with the audience. The way to be effective with big data is to be effective with visual communication through data visualizations and engagement.
Data storytelling is not always a classic story format, with a protagonist, antagonist, problem, and solution. That is one formula for storytelling. But I am not always sitting in an audience expecting a story with a happy ending. With data, sometimes I need to evaluate options and make business decisions—this is the probabilistic world prediction, prescriptive and cognitive computing with big data. I need to have everything in front of me—visually—to have the final decision and take action. If I am going to do discovery, I want to quickly see all the data and options on the screen and how they are related. And in the cognitive computing world, data visualizations are a critical component of helping machines make critical decisions quickly through simplifying visual communication and focusing on interaction efficiency—but not without the user assessing and validating the results.
In order to communicate visually and correctly, whether in storytelling or visual discovery or whatever other visual communication method, it is absolutely necessary to do so without getting in the way of the user and what they are trying to understand from your message.
Imperative 4: Visual communication as a core business competency
When we talk about architecting for discovery, we are talking about managing data in all its different formats. When we talk about data visualization as a core competency, we are talking about user experience and technical design elements. When we talk about enabling visual discovery or visual analytics, that is bringing purpose to both. In order to realize the full potential of big data and its new capabilities, you have to unlock visual design and communication as an essential competency at every level of author and audience within the enterprise.
Consider this: You can read through a page of text, but what if you could have a really good visualization that you can understand 100× faster? Now imagine if your company could consistently communicate this way. The impact would change the way you do business—speeding the business strategy itself. This is not to say that data visualization is the only form of business communication, but done properly it is an effective, efficient way to discover and comprehend data and insights.
That is why this is so important. We are in the big data world and there is no going back, primarily because your competitors are not going backwards and your users have come to expect a new paradigm.
While proper technical visual design was in the past relegated to creatives and designers, it is now part of every business. Does everyone in your company know how to do visual design correctly? And once they know that, do they know how to communicate visually to deliver the point or solicit interaction—whether it is to convince me, help me discover, or help me make a decision? That is why the storytelling aspect is not just a hot trend; it is part of the big picture.
So, organizationally how do I create a company culture that recognizes and embraces these principles when you already have the tools? How do I make this part of the fabric of my business so everything runs 100× more efficiently and effectively? You cannot work with big data and work with numbers in a spreadsheet. You cannot have the best data visualization tools and let everyone do something different without proper training. As business leaders, change agents and champions in data driven organizations, you simply need to know.
That is why it is a visual imperative, and why the content of these pages is so relevant for every business leader, every analyst, and every businessperson today.
CEO of Radiant Advisors