This blog was first published on Forbes.
With the World Cup upon us, it’s an apt time to draw inspiration from soccer. In 1950, Charles Reep, an accountant, attended every game of the Swindon Town soccer team’s season, tracking events and recording statistics. He analyzed his data and concluded that long passes were the most effective way to outscore opponents. He would later be proven wrong. Reep came to the wrong conclusions in part because didn’t have the tools or data — player tracking software and expected goal algorithms, for instance — that modern soccer teams and analysts have.
Today’s companies can’t afford to come to the wrong conclusions like Reep did. They also have no excuse for doing so: available to them are state-of-the-art data collection and analysis tools. One such tool is the data lake.
Data lakes, when used correctly, improve an organization’s business and operations performance. They deliver benefits in the form of better responsiveness, faster time to market and improved competitive advantage, as well as better alignment and communication in internal processes. These benefits don’t come automatically. To realize them, organizations should institute processes and choose technologies that allow businesses to access and leverage insights within data lakes.
Data Technologies Are Changing; Your Analytics Approach Should, Too
One thing Reep did not suffer from was the paradox of choice. More choices can cause more anxiety when making a decision, and Reep had only his pencil and notepad to choose from. Today’s organizational leaders must contend with an explosion of data and an ever-growing selection of data lake technologies. These offerings continue to evolve, requiring different approaches to analytics and business intelligence (BI), like BI on Hadoop, to elicit value from data.
Go Beyond Transaction Data
The batch processing we once used to feed static data warehouses is no longer sufficient on its own. Neither are the analytical tools that assume the underlying data is static. If we want to avoid the fate of soccer teams who followed Reep’s conclusions, we need to ensure our data lake architectures can process and analyze the accelerating streams of dynamic data we see today.
The size-constrained data warehouse, the technology predecessor of the data lake, typically contained transactional finance, sales and operational data. Now, businesses have more digital channels and means to collect observational data from machines, which means the data itself is changing.
Underlying data-management technologies are also changing to keep up with the expansion of data variety. Hadoop deployed on-premises was the primary platform for many of the first data lake deployments. Now, NoSQL technologies and cloud-based deployments are also becoming common. Cloud object stores, such as Amazon S3 (an Arcadia partner) coupled with Spark processing, are also increasingly considered viable alternatives. To realize value with their technology choice, users must be able to access and analyze the information available in data lakes easily and efficiently.
Data lakes can handle not only this greater volume, but they can also help store and analyze a wider variety of data. To maximize the value of data lakes, organizations should broaden their horizons to include other forms of data, such as interaction data from customer service centers, event and machine data from operations processes and, finally, streaming data.
Leverage External Data Sources
Whether analyzing soccer games or building balance sheet projections, context is everything. That’s why it’s important to tap external sources, so we don’t perform analysis in a vacuum. Analysis of external data, such as social media streams or demographic or weather data, can help an organization improve the performance of products and services. Combining these data sources with internal data for analysis offers a more complete picture of customer and market dynamics, enabling better-informed business decisions.
Enable New Types Of Analytics
Imagine if the soccer world continued to believe long balls were the key to winning games or performed manual tracking to gather information. It is unlikely the Beautiful Game would look as pretty as it does today. Since Reep, the sporting world has evolved, introducing new forms of analytics. As in soccer, data lake investments should enable analytics that extend beyond those currently being used.
With the larger volumes of granular data stored in a data lake, organizations can perform analyses at a much finer level of detail than was previously possible even just a decade ago. Such detailed access allows employees — and not only information technology (IT) or technically inclined employees — to follow and understand trends in data within a single environment. By providing this level of detail, data lakes also enable forward-looking predictive analytics using machine-learning and artificial intelligence techniques.
Provide Self-Service To Business Users
None of these suggestions will be helpful if business users cannot easily access data lakes to perform analysis. IT is often an overburdened department anyway, so introducing new tasks for them will only back up the bottleneck that likely already exists. Still, many organizations hesitate to give business users data access because of governance and security risks. It’s possible to build provisions into data lakes that keep sensitive data unavailable to certain users, for example. To avoid time-consuming analysis processes, design your data lake to extend self-service capabilities and reduce IT bottlenecks.
Go With The Flow Or Be Left Behind
Whether in sports or in business, the types of data are growing and the rate at which data flows is quickening. Organizations that do not adjust their strategies and reconsider their technology options will not remain competitive as data environments evolve. In order to stay ahead of these changes, leaders must adopt tools and processes that are flexible and adaptable. With this approach, your organization can continue to leverage data to improve its performance even in the face of change.