April 26, 2018 - Paul Lashmet | Big Data Ecosystem

Democratization of Advanced Analytics

Co-written with Peter Memon, Managing Director – Global Head of Emerging Technology at Synechron. This article was originally posted on Synechron

Mind the Gap between Technology and Business Intent

Solutions to complex analytical problems often rely on the craftsmanship of highly-specialized experts in data science, machine learning, and other branches of artificial intelligence.  The extent to which advanced analytics creates business value today is still limited (especially for businesses like financial services) because of a few key reasons, all of which also drive up the cost of monetizing a company’s data in the end. These include:

  •   Finding the right combination of advanced analytical skills and domain expertise doesn’t meet industry demand.
  •   Complex technical architecture as well as complex underlying data structure unique to specific relevant data-collection processes.
  •   Businesses often have in place a complicated set of tools – oftentimes legacy systems even – that make incorporating other newer [perhaps more complex] tools extremely hard to integrate.

The combination of these three primary factors present a significant challenge to generating positive return on investment (ROI) and establish a sizeable gap between the ability to implement sophisticated analytical tools and the actual intent of the business.

Moreover, the select few who understand how to create and run the machines of advanced analytics are usually not the domain experts who understand the historic relevance of a business scenario (e.g. time series trading data) and who can quickly discern nuanced disruptions of current events (e.g. end of day in equities vs. the 24/7 FX market) to flag future opportunities.  The business users who will be successful at gaining meaningful insight from data science will be those who are able to democratize it to scale their insight across a broad plane of opportunities and focus it based on an understanding of technical drivers and business demands and how they function not just in an innovation/R&D environment but at scale in the enterprise. This ability would help bridge the gap between implementation of advanced technology and business intent.  However how can the technology side meet the business half-way?

Moving Beyond Traditional Business Intelligence Tools

As financial institutions look to gain deeper business insight and intelligence, BI tools of the past won’t provide the level of depth these businesses are seeking, and therefore, many organizations are exploring how data science can help. The democratization of analytics, as described below, is a current process that has emerged and that helps to bridge the technology and business application gap – from the technology side.

Advanced analytics that target discerning customers with customized financial products, predict the best price on one of millions of active trades, or correlate a variety of alternative data sources to enhance trading strategies is becoming the norm.

These tools often do not fit into the intended implementation because they have not evolved fast enough to keep up with the newest wave of data type or source.  They often rely on data that has been generalized into a predefined data structure (i.e. data warehouses, data marts) while the data that provide new opportunities are often large and messy, originating from disparate sources in a variety of formats. The use of complex alternative data sources like the Internet of Things (IoT) streaming data, unstructured documents, and pedestrian/vehicular geolocation data to enhance investment strategies adds to the complexity of leveraging advanced analytics. The business must be armed with tools that can evolve and morph as quickly as new data types, new sources, and increased frequencies of this data become available to them.

Experience the Democratization of Advanced Analytics

The ideal result of the democratization of advanced analytics is to make a range of data science resources, sophisticated platforms, and tools available to subject matter experts who can apply them to their business challenges in a self-driven way, and when they deem relevant.  The process is picking up speed with three distinct types of platforms.

  •      Crowd source data science:  Businesses can pose complex data challenges to a community of data scientists who then compete to provide the best solution for a financial award.  The business can benefit by engaging data science resources on an as needed basis.  Several platforms exist that enable this concept and have solved a variety of problems including: predict the daily spot price for copper, identify new products for unique minerals, forecast economic outcomes.
  •      AI to do AI:  In this scenario AI begets AI by creating a variation of AI algorithms to solve a problem from which another set of AI algorithms then determine which of the first set is the best. Algorithms generate the models without requiring a data scientist and iterate over tens of thousands of models before they arrive to the best solution.  These include such versatile tools as DataRobot and H2O’s Driverless AI.  This type of platform increases the bandwidth of a data scientist by automating the generation of statistical models.
  •      Let AI provide the answer:   You are a business user with a limited data set and want to ask questions in a way that is more ‘human’ without the benefit of having a data scientist with a PhD in Statistics next to you. Bayesian statistics and generative models are increasingly coming to the rescue.  Maybe you want to ask, “How much does a 33-year-old man with two jobs, three children and two houses make?”. Certainly not a lot of data points on that! Using advanced Bayesian statistical methods combined with generative models a data scientist can answer to that question.  There are now a class of platforms such as Empirical that allow users to do exactly that.  For smaller, sparse data sets these methods are an excellent choice.

Conclusion:  It All Comes Down to the Data and the Questions Asked

It is clear that businesses are asking increasingly complex questions that traditional BI tools cannot answer, and while data science seems to hold the answer, even advanced financial institutions still have a gap between their technical execution and getting to a place where they can make those powerful analytical capabilities available to business users. Democratization of analytics is about increasing the accessibility of sophisticated analytical capabilities by putting the power of these methods into the hands of the business users.  The transition to a hands-on, self-service approach where the competitive edge lies in giving every business user access to powerful tools will change the way business users understand their businesses, without requiring a large team of data scientists. There are a number of tools and techniques available in the marketplace today, and if businesses put democratizing analytics and institutionalizing it on business-side decision-making, they will be able to have significantly more powerful data driving their businesses.

Related Posts