Big data deployments are seldom quick or easy. In some cases, many months can pass before you begin proving value. After all, big data – and the analytics on that data – come with a variety of challenges and considerations.
First, how will you collect data? A common objective of those architecting a big data environment is to take advantage of as many data sources as possible. You will want to collect a mix of structured, unstructured, and multi-structured data. Some sources will provide streaming data, others will be historical data. You’ll need to collect data in a variety of formats. If you don’t get data collection right with your initial data sets, you’re going to struggle when adding more data or more complex data. That likely means you won’t even get a chance to explore other data sets.
At the same time, you must consider what backend will support all that data. Big data quickly becomes very expensive when you try to shoehorn it into a traditional platform, and you may not be able to shoehorn it at all. This is why modern technologies like Apache Hadoop have proven strong candidates for a big data backend.
Next, how will you provide access to this data? Depending on your use cases, maybe raw SQL (or SQL-like) access is important. Low-level, code-heavy environments for analytics have been commonly used. But we are seeing growing interest around visual analytics capabilities which are critical for both data experts and non-experts to develop and communicate valuable insights.
Ultimately, you are striving to enable a broad list of use cases across a variety of data types from many different sources. Your big data environment should enable marketers to create more targeted, personalized offerings, and it also needs to enable data scientists to build artificial intelligence and machine learning models that power groundbreaking applications. You need one ecosystem for a variety of audiences.
Finally, you need to address how you will manage the system. Big data talent is in limited supply, which means you’ll likely have to invest heavily in training and/or hunt for the existing experts.
It’s critical to think about all these considerations in the context of your use cases. After all, Gartner estimates that 90 percent of deployed data lakes will be rendered useless through 2018, as they’re overwhelmed with information assets captured for uncertain use cases.
Big Data as a Service
Fortunately, the “as-a-service” approach that is so prevalent in the software we use every day has made its way to big data with managed services like Cazena, which today announced its App Cloud. The App Cloud makes it easier for companies to deploy the Cazena big data as a service platform along with solutions for data ingestion and analytics, so you can deploy a complete managed big data ecosystem in the cloud in a single transaction.
Arcadia Data is pleased to be a key partner in the Cazena App Cloud (which also leverages our strong partnerships with Cloudera and StreamSets). Cazena eliminates many of the challenges in deploying your own data lake or big data platform. The Cazena service is built on top of Cloudera in the cloud, but because it is offered as a service, you don’t need any special cloud or dev-ops skills to get up and running. This reduces your dependency on big data talent.
The App Cloud includes both Arcadia Enterprise and the Cloudera Data Science Workbench. The Cloudera workbench enables data scientists to leverage Python, R, and Scala directly in their web browsers to create and manage analytic pipelines. Arcadia Enterprise extends this by offering powerful visualizations for use by a wide variety of users, especially business analysts. Our technology is architected for big data and cloud deployments, so it’s a natural fit for us to partner with Cazena to make big data accessible to more organizations.
If you’d like to perform visual analytics at scale but don’t have all the resources to deploy or manage your big data ecosystem, consider the App Cloud and its managed, cloud-based approach to big data deployments. Contact us if you want to learn more about how Arcadia Data is helping businesses get value from big data.