January 18, 2019 - John Thuma | Big Data Ecosystem

Arcadia Data Agrees: Use Materialized Views!

Zoomdata recently published a blog post detailing their use of materialized views as a means to “turbo-charge BI.” In the blog, Ruhollah Farchtchi, CTO at Zoomdata, discusses how traditional BI tools and methodologies are failing to keep up with the needs of big data. He is right, but of course materialized views are nothing new. Materialized views have been around for a long time and are well known to anyone familiar with relational database management systems. Simply put, a materialized view is a named and persisted database object from the output of an SQL statement. Think of it as a snapshot table that exists as a result of a SQL query. Oracle introduced the concept in version 8i, and now many legacy database applications support them. Materialized views require an understanding of SQL, security, and scheduling tools to make them work properly.

Arcadia Data agrees! Leveraging the power of materialized views provides a huge speed advantage and is a core component of our Smart Acceleration technology. The output of Smart Acceleration is a materialized view we call an analytical view. Analytical views are very fast! In a recent ESG report, Arcadia Data demonstrated an 88x performance benefit over external BI and middleware solutions. Another advantage was massive improvement to concurrency. Check out the Arcadia Data ESG report for more details.

Arcadia Data’s patented Smart Acceleration feature not only provides huge performance and concurrency benefits, but it also provides many other capabilities as well. There are five key factors that make Smart Acceleration smart. They include: zero coding recommendation engine, zero data movement, zero object redundancy, secure by default, and you can even use your traditional BI tools like Tableau, Qlik, and others. Let’s address each of these advantages.

Zero Coding: Built in Materialized View Recommendations Engine
Arcadia Data’s Smart Acceleration takes the guesswork out of what materialized views to create. We fondly refer to this as having a “DBA in a box” because it removes the burden from a DBA. Our recommendation engine looks at your dashboards and visuals, then constructs the necessary analytical view for you. You don’t need to know how to code the materialized view either. Our Smart Acceleration technology does all the work for you. It is all point and click, and you never see one line of code. Smart Acceleration also provides a simple scheduling interface to keep the materialized views up to date. Check out this video to see how easy they are to create.

Zero Data Movement: Never Move Your Data!
Arcadia Data does not move data into Spark, a fancy BI server, or to an OLAP cube. We build the analytical view where your base data is located. That means your data does not move off the data lake. Forrester refers to this approach as “in-data-lake BI,” where BI processing is brought to where the data natively resides. As your data lake scales out, Arcadia Data scales with it; you don’t have the physical limitations like you do with BI middleware solutions. BI middleware solutions only scale up, so if your server is almost full, you might be out of luck! Zero data movement means minimizing latency, reducing security risks, and aligning to a single version of the truth guiding principle.

Zero Redundant Objects: Never Create Two Identical Materialized Views.
One of the biggest problems in relational database materialized views is that developers create multiple copies of the same data everywhere. As a BI developer, I too am guilty of this! I admit I have built many materialized views that are exact copies of one another. Smart Acceleration is…well…smart! It never creates duplicate materialized views. Any Arcadia Data dashboard that attempts to use that dataset automatically gains benefit from the object too. You don’t have to think about it, it just works.

Secure by Default: No Additional Security Footprint
Security and data protection is top of mind for everyone these days. Traditional materialized views require knowledge of how the object is secured. With Arcadia Data, whatever policies you have on the base data is automatically enforced on the analytical views created by Smart Acceleration. Smart Acceleration takes advantage of LDAP authentication and authorization. If you are using Apache Sentry or Ranger to protect your data, Arcadia Data automatically enforces them on all objects, including the materialized views. This defense in depth feature is critical to protecting your data landscape.

Traditional BI Tools: Traditional BI Tools are Welcome!
Traditional tools cannot do what Arcadia Data can with respect to Smart Acceleration. This doesn’t mean we can’t use them to connect to the analytical views created by Smart Acceleration. They are welcome to the fast big-data party! Analytical views (aka materialized views) created by Smart Acceleration are in Apache Parquet format. This means that tools like Tableau, Qlik, and other traditional BI tools can take advantage of our Smart Acceleration product without a pesky middleware solution.

Hats off to Ruhollah from Zoomdata and his blog post that validates our approach to Smart Acceleration and materialized views. Arcadia Data has first mover advantage as the industry’s leader for in-data-lake BI. Arcadia Data provides a complete solution for analytical views. Smart Acceleration is just one feature of many that makes Arcadia Data interesting. We also provide the first and only Confluent KSQL integrated solution for real-time streaming analytics. Our search-based BI tool can open up natural language query. Arcadia Data makes big data fast! It is an exciting time!


Related Posts