Big Data promises a wealth of riches in the form of new insight into operational processes and business opportunity. But for many organizations, the ‘how to’ remains “a riddle, wrapped in a mystery, inside an enigma.” [1] According to Gartner research, Hadoop skills gaps emerged as a major adoption inhibitor for close to 60 percent of US businesses in an 2015 survey, while a 2016 study found a lack of holistic strategy that can support appropriate investment in infrastructure, based on clearly identified business outcomes behind an inability to turn Big Data pilots to production.
To help address these gaps, Cloudera and CenturyLink Global IT Services are working together to deliver Big Data-as-a-Service, a comprehensive offering aimed at simplifying deployment and at helping customers to derive value from all their data assets. Today, the relationship between the infrastructure/communications services provider CenturyLink and Cloudera, provider of a modern data management and analytics platform based on Apache Hadoop targeted at the enterprise, consists of IT managed services that the partners claim speed time to deploy Cloudera Enterprise and simplify management of Hadoop operations.
Based on the ‘eat one’s own breakfast’ principle, CenturyLink developed the expertise needed for this combined offering by building out a very large scale internal data hub on the Cloudera platform. The organization’s goal was to execute on what Andrew Clyne, chief data officer, CenturyLink Cognilytics, called “Agile Analytics best practices” – combined reporting and analysis of transactional structured and unstructured data held in the same repository. “For example, you can look at correlations or create models across data sets ranging from information used by line of business for networking operations all the way through marketing and sales. The extent to which you can add data sets together in a central place, you increase the value of all the other data that you have already put into the data hub,” Clyne explained. So far, CenturyLink has defined 17 use cases for the concurrent analysis of the company data that will drive value through cost savings, cost avoidance, or new revenue opportunity – and two of these (marketing and operational analytics cases) will deployed by year end.
At the core of the Agile Analytics lies the enabling Hadoop technology. In contrast to traditional, waterfall-driven methodologies for establishing a data warehouse and layering on BI applications, Clyne argued that “with a data hub, you are able to very quickly pull the raw data into a Hadoop platform, and spin up combinations of the data as if you are spinning up a data mart. You are able to look at the results of what you have produced much faster than you can with a rigid data warehousing approach, lifecycle and model.” Leveraging Cloudera technology, CenturyLink has developed a process by which it is able to spin up the data, prototype the analytics and prove out its hypotheses across large volumes of data collected over months, and deliver the visualizations needed – as in Agile software development – to engage business within 90 days. Clyne notes that a lifecyle process that covers all data requirements and which is based on traditional data warehousing methodologies often can consume a year or so to complete.
In scoping its data hub project, CenturyLink evaluated a number of Hadoop distributions (and continues to provide support for MapR, Hortonworks and open source native Apache Hadoop) but chose to build on Cloudera for a number of reasons. First was access to operational management tools which come with the distribution that would deliver TCO savings over time, but CenturyLink was also attracted to other value added capabilities (security controls, encryption at rest) that support enterprise grade deployment, and additional new features that could be incorporated to create a “very “powerful” offering, such as an analytic database with Apache Impala or Apache Kudu for its own customers. And in building out and deploying the data hub, CenturyLink also built accelerators and best practices that it has been able to leverage for its customer offering: “we were taking what we had before – our own Cognilytics Hadoop expertise – and combining that with the best and brightest of Cloudera, using that to enhance our existing go-to-market service for Cloudera in our managed services.”
CenturyLink provides support for Cloudera in many modes – for on-premise implementations, for Cloudera deployments hosted in CenturyLink data centres, or in other clouds, such as Microsoft Azure or Amazon Web Services (AWS). Priced on a subscription basis, the offering covers four service tiers: support for the basic infrastructure; management of the Hadoop ecosystem, including patches, certifications, upgrades, the addition of nodes to the cluster to accommodate data growth, and help with administration of Spark, Flume, Kafka, and Sqoop streaming; data management, including ingest, management, data quality, controls – essentially ELT/ETL for the platform; and a block of hours that allow clients to tap into the expertise of CenturyLink’s Hadoop architects for help with integrations, architecting and onboarding new use cases, or the creation of an end-to-end solution.
Since the launch of the Cloudera managed service offering, Clyne said the company has seen a lot of interest in tier two services, particularly among the Fortune 1400 – who are not likely compete successfully for scarce Hadoop skills. Typically, the company is able to begin with this type of basic support, and then extend into analytics, delivering data science and machine learning expertise, which are also in short supply in the marketplace, to help customers find value in their data. With the “Quick Start” program, for example, CenturyLink will help the customer prioritize use cases, and focus on having one of these up in pilot mode and ready for production within 90 days. As Cloudera CEO Tom Reilly observed, “CenturyLink and Cloudera have a history of outstanding customer service and innovation, and we’re pleased to be helping deliver a data management and analytics platform that drives significant business benefits. CenturyLink’s Big Data-as-a-Service offerings will make concepts like advanced analytics, machine learning, and real-time data more available and accessible to more people. And that’s where true business transformation happens.”
[1] Winston Churchill, speaking on Russian foreign policy interests in a radio broadcast in 1939. http://www.churchill-society-london.org.uk/RusnEnig.html
