Do you know what big data is? Do you excel in big data analytics? What if I told you that you don’t actually know what big data is about? You might think you do, but you don’t.
Or maybe you DO know. Interestingly, I’ve found that many people have a misconception about big data. Think about it: do you ever question the validity of that term? Do you ever wonder if big data is a real thing? If so, do you also question the need for terms like metadata, relational data, unstructured data, and time-series data? After all, they’re all just data, right? Isn’t that a sufficient descriptor for these things?
If you agree there’s a very real distinction between metadata and time-series data, then you’re on to what I’m getting at: the term big data also has a purpose. When I first started dealing with the term big data many years ago, the point was clear to me. It was about a problem that enterprises were facing, with their stockpiles of data they wanted to leverage. It wasn’t about some newly suggested benefits of your existing data sets, but rather about a shift in the characteristics of data that are now giving you headaches. We tried to capture that in our fun new video (more on that later). Back then, I was working for a technology company that built software that addressed this growing problem, so big data was a natural term for the challenges our customers wanted to solve.
Many people first heard the term when they actually had no exposure to real big data. So to them, big data was theory or even fiction. It was a made-up word that represented something they did not see. This perception that big data was made up soon turned to a dismissal of the term as marketing hype. The most cynical of the bunch would say the term was only used to fool people to buy more software. But at some level, we should be able to agree there are some tasks that relational databases do better than any other data platform (like running consistent transactions), and there are tasks they are not ideal for (like petabyte-scale data analysis, especially in a cost-effective way).
To me, the clearest example of one’s misunderstanding of big data was the insistence that the “three Vs” definition of volume, variety, and velocity is actually “four Vs.” Here’s the problem: the initial three Vs referred to challenges and were not merely labels, and the commonly cited “fourth Vs” were always descriptive labels. If we viewed the three Vs as bad, then the implication around big data made sense: data with excessive levels of the three Vs is hard to manage. Specifically, it is hard to manage with traditional platforms like relational databases. Therefore, you needed to handle big data in a different way, with different technologies. “Fourth Vs” like “veracity” and “value” added nothing to the definition of big data because you didn’t have to do anything different to deal with those Vs.
So now that I’ve given you my unsolicited input, I want to share something more lighthearted. We’ve personified big data as an annoying bully that can give you grief. This personification is in our big data bully video and calls out the challenges that IT and business professionals face with big data. But rest assured, the bully will not win, because, fortunately, more and more solutions are arising to help you deal with the big data bully.
From the Arcadia Data perspective, we’re here to help companies deal with their big data bully problem by giving the right tools to business analysts and business users. Our stance is simple: just as you can’t easily solve big data management with a traditional data platform, you can’t solve big data analytics with traditional BI tools. Arcadia Data built a modern BI platform to increase agility by simplifying several resource-intensive tasks in the analytic lifecycle (moving data, modeling data, building visuals, and performance analysis/modeling) required for big data systems. If you continue to use your traditional BI tools, then you’d better watch out for what comes out of the elevator (see image above).
If you want to get hands-on experience with the data visualization we provide, download Arcadia Instant for free and explore our powerful user interface on your desktop. You only get a subset of features in Arcadia Instant versus Arcadia Enterprise, so when you’re ready to take on the big data bully directly, be sure to reach out to us. No one likes a bully. We can help.