February 6, 2017 - Susan Rojo | Big Data Ecosystem

What’s the Difference Between Big Data and Business Intelligence?

Whenever the term “business intelligence” comes up, it’s a fair bet that “big data” isn’t far behind. While to an experienced knowledge management professional the difference will be clear, other team members unfamiliar with these terms can possibly get confused. To further complicate the matter, IT managers and CIOs sometimes use the two interchangeably.

For example, it’s not uncommon to hear something like, “We need to bring in big data applications to solve our supply chain bottlenecks.” In reality, that would be a case for bringing in business intelligence tools, not the more general big data applications or data platforms. The problem is that even a small misunderstanding in terminology at an early stage can have a huge impact on your project in the long run.

Here’s a closer look at the unique characteristics of big data in contrast with business intelligence, as well as the areas where the two overlap.

Big Data Defined

It’s hard to believe, but more than 15 years ago, Gartner specified what turns regular data into “big data” in terms of the three V’s:

  1. Volume – This is big data’s most identifiable aspect. It refers to the mind-boggling amount of data generated each second by users, sensors, and servers all over the planet. In 2015, Forbes reported that more data was created in the two prior years than all the data produced over the entire history of the human race.
  2. Velocity – The internet has massively accelerated the speed that data travels from point to point. This covers everything from emails to real-time monitoring, to online financial transactions. Consider that more than 100 million emails are composed every minute, and then consider the speed of data moving across social media networks and chat apps.
  3. Variety – Nearly all the data generated is in an unstructured format, including video, information coded for proprietary databases, region-specific data, voice data, metadata and many other types that require normalization.

Over the years, large enterprises like IBM have expanded the definition to include five V’s, incorporating:

  1. Veracity – This is a highly significant factor that only really became apparent as data analysts began to work with big data. The trustworthiness of the data source and the time necessary to clean up the data before it could be used deeply impact how useful the data is to an enterprise. The length of time between when the data is collected and the time it is analyzed contributed to a new plague of “data rot.”
  2. Value – As the above suggests, the complexity of converting raw data into useful insights and actionable conclusions influences whether it should be considered big data or merely a big headache.

Really, there’s no reason to stop at five. The number of Vs that could be added continue to increase as more people begin exploring the possibilities. Data experts have suggested adding vulnerability to stress the growing security concerns over the use of big data and viability to highlight the fact that not all data, even if accurate, will have a meaningful impact on desired outcomes.

In fact, that final point is a good introduction to where business intelligence comes into the picture.

The Path to Business Intelligence

The first use of the term business intelligence goes back 150 years to the end of the Civil War. Today, it still retains its original meaning of data that allows a business to react swiftly to changes in the market. Only the technology has changed.

In 2017, business intelligence incorporates applications in data warehousing, extract transform and load (ETL) tools, online analytical processing (OLAP) and the open source framework for distributed data storage and processing known as Apache Hadoop.

Today, the most advanced business intelligence systems are capable of handling processes like:

  • Micro-Segmentation Analytics – This allows analysts to slice up any population, such as customer or market data, into entities for a better understanding of their behavior at a fine grain level. It also means data specialists are easily able to convert massive data dumps from external sources into detailed targeting profiles.
  • Direct Visualizations from Cloud-based Storage – Data visualizations are the heart of better decision-making. Most people need to see data presented in a format that immediately communicates the importance of the information and implies the best response. The facility for running visualizations directly inside the cloud-based storage lowers the time and resource commitment of making a copy of the source data and storing it on premises. This puts the most important big data discovery tools in the hands of smaller organizations and less technical managers.
  • BI Dashboard – A well-designed business intelligence (BI) dashboard, or application, collects the most important big data analytics tools into a single interface so managers can review the state of the business at a glance.

A web-based UI makes it easy to build, filter and iterate rich visualizations that can be embedded and shared securely with anyone anywhere.

Big Data vs. Business Intelligence

Practically speaking, big data is the term that should be applied to discussions regarding information sources and strategic directions. Business intelligence is the term to use when discussing systems that help leaders make better decisions for a specific company at a tactical level. That’s a good rule of thumb for using the right term under the right circumstance and eliminating misunderstandings down the line.


Related Posts