Chapter 6:
Making Big Data Actionable

Today’s “Brave New World” of big data BI offers the potential of visualization to create new perspectives and insights. However, many of these initial offerings are still limited to data that has been extracted and prepared, often by costly data scientists. To realize the full potential of big data, business intelligence applications must evolve to support visualization directly from all data stores. They must eliminate the dependence on data cubes and predefined views while also integrating real-time and streaming data from sources like intelligent devices in the Internet of things.

Forrester Research has said that “customer obsession and big data call for more BI muscle,” observing that nearly one-third of organizations now store and process more than 100 TB of structured data. “To process and analyze all this data efficiently and effectively, application development and delivery pros working on BI initiatives need highly scalable and distributed BI platforms and flexible and agile technologies,” wrote Principal Analyst Boris Evelson.

The Next Generation of Business Intelligence

The change begins with the client. Since many legacy desktop BI tools failed to offer capabilities to support real-time collaboration, IT organizations had to ensure there were enough server resources to share analytics. A small implementation could carry a hefty price tag as it becomes deployed enterprise-wide. Since desktop BI clients leverage local copies of data downloaded from a server, they tax network bandwidth and introduce version-control problems, not to mention additional security risks as well. Further, many BI tool users simply use them to extract data, while relying on spreadsheets and other tools for more advanced analysis and formatting. Users then typically exchange these reports by emailing them to each other, exacerbating the data duplication and version control nightmare.

This is not to say that these traditional BI tools are not used for analysis or building visualizations. In many instances, a company’s IT organization is tasked with building complex analytical applications such as balanced scorecards and dashboards using native functionality as well as APIs offered by these vendors. However, these pre-canned views are often designed to provide little or no ability to conduct ad-hoc queries or even “what if” questions that were not anticipated when the cube or view was created.

The need for BI servers should also be revisited in light of the limitations they place on working with unstructured data. Extracting and transforming data is a time-consuming process that is inconsistent with today’s need for rapid decision-making. ETL also requires skills that are in short supply. And the fact that data scientists are often responsible for data wrangling only worsens the situation. Accenture found that 80% percent of new data scientist jobs created between 2010 and 2011 had still not been filled two years later. Numerous studies have reported that the average salary for data scientists in major markets exceeds $200,000. That doesn’t include the tax that BI servers place on IT infrastructure and budgets. Hadoop and the cloud has revolutionized the economics of data. Why not extend those benefits to business intelligence?

The Browser Is the New PC

It is highly unlikely that the minds that developed what is now known as today’s Internet would have imagined how it has impacted our lives. With the increased proliferation of smartphones and other devices simplifying access to this “information superhighway” coupled with the demand for a more uniform “user experience,” various companies have attempted to find ways to leverage these advances.

It is highly recommended that BI clients should leverage the native functionality provided by one of these modern browsers which support advanced programming languages like JavaScript and HTML5. Among the powerful new features HTML5 provides are the ability to accept non-HTML data, store and operate upon data locally, play video and audio without the need for plug-ins, display two- and three-dimensional graphics, and optimize for local processing resources, all of which would prove useful for an individual looking for more than text.

By incorporating these native capabilities, browser-based BI tools reduce the need for application installation and maintenance. This in turn significantly reduces licensing fees and IT support costs in both the short as well as long term.