Visual-based data discovery, a defining feature of the modern business intelligence (BI) platform, began in around 2004 and has since transformed the market and new buying trends away from IT-centric system of record (SOR) reporting to business-centric agile analytics. Modern BI and analytics platforms are characterized by easy-to-use tools that support a full range of analytic workflow capabilities and do not require significant involvement from IT in order to predefine data models upfront as a prerequisite to analysis (including at enterprise-scale deployment).
Moreover, as the visual-based exploration paradigm has become mainstream, a new innovation wave is emerging that has the potential to be as disruptive as (or more than) visual-based data discovery has been to the previous semantic-layer-based development approach of traditional BI and analytics platforms. Smart data discovery — introduced by IBM Watson Analytics and BeyondCore (acquired by Salesforce as of September 2016) — leverages machine learning to automate the analytics workflow (from preparing and exploring data to sharing insights and explaining findings). Natural-language processing (NLP), natural-language query (NLQ) and natural-language generation (NLG) for text- and voice-based interaction and narration of the most statistically important findings in the user context are key capabilities of smart data discovery.
“By visualizing information, we turn it into a landscape that you can explore with your eyes, a sort of information map. And when you’re lost in information, an information map is kind of useful.” — David McCandless TED talk
The expectations of the enterprise users around intuitive, user-friendly interfaces are rapidly shifting forcing enterprise BI firms and application developers to react. The focus is on enabling users to explore and analyze data with simple drag-and-drop operations.
People within organizations have traditionally accessed data via static reports from enterprise applications and business intelligence platforms maintained by IT departments. These systems, predominantly designed and built in the 1990’s, are generally heavy, complex, inflexible and expensive. As a result, business users are forced to depend on specialized resources to operate, modify and maintain these systems.
Faced with these challenges, many knowledge workers today rely on spreadsheets as their primary analytical tool. While spreadsheets are widely available and easier to use than traditional BI platforms, they have a number of limitations. Spreadsheets are not generally designed to facilitate direct and dynamic data access, making the process of importing and updating data manual, cumbersome and error prone. In addition, spreadsheets are not built to accommodate large data sets and offer limited interactive visual capabilities, thereby reducing performance and limiting analytical scope and insight.
Research has shown that when data is represented graphically, we use less cognitive resources to make a decision and retain information better, thus “speeding time to insight.” Users can drill down into datasets with simple queries, and discover patterns and trends presented in highly visual interactive dashboards. “By presenting data in a graphical mode, outliers and anomalies can be spotted almost immediately, versus the time taken to pour over rows and columns.”
Data discovery is not a tool. It is a business user oriented process for detecting patterns and outliers by visually navigating data or applying guided advanced analytics.
Visualizations make use of our brains’ pattern recognition capabilities to digest information at a glance or even pre-attentively. Users are better at finding insights and detecting outliers if data is presented in charts and graphs on one page, versus being buried in data tables spanning multiple pages.
Visual analysis is an important feature that is increasingly being sought by enterprises seeking more efficient ways for decision-makers to absorb and act on data.