InsightaaS: Analyst firms play many different roles in the IT industry. Most offer some level of opinion; some offer data, often instead of informed commentary. A select few are able to combine real research with perspective that helps companies to adopt and use leading-edge technologies.
451 Research is part of this select group. 451 the clear leader in researching and reporting on cloud pricing – work that is critical to advancing cloud to the centre of IT/business infrastructure. Analyst Owen Rogers has built the Cloud Pricing Index that tracks costs and factors impacting costs – research that has been featured on InsightaaS in the past. In this report, Rogers takes a somewhat different approach, looking at the capabilities and tools that support cloud pricing – and which enable IT leaders to track and control usage and expenditures, allowing them to “monitor and optimize on their own terms.”
Cost reporting is not simply a nice bonus for enterprises curious about their cloud spending. As hybrid and multi-cloud deployments become commonplace, understanding and optimizing cloud expenditure will become increasingly critical to enterprises. Service providers should ensure that end users have access to this data so they can make business decisions on best value across their IT estate and the whole enterprise.
The 451 Take
Cost and usage reporting will become more important to enterprises as cloud takes on a more fundamental role in their operations. As the use of hybrid and multi-cloud becomes more ubiquitous, understanding, managing and optimizing consumption across these different providers and models will become vital. A GUI to show this information is a good start, but ultimately APIs will be needed to allow CIOs to monitor and optimize on their own terms. Service providers with big names on their target list should consider developing this capability sooner rather than later. Enterprises should consider how they will manage expenditure as their deployments grow – with poor financial reporting capability, CIOs might struggle to keep track and keep control.
We hear from all quarters that business challenges, not technical ones, are the main barriers to cloud adoption. Setting up a cloud application, or even a private cloud, is relatively easy compared with the challenges of keeping it compliant, documenting change and integrating it with existing processes, polices and cultures. 451 Research’s Voice of the Enterprise previously found a whopping 74% of roadblocks were down to non-technical issues. It might not be surprising to hear that enterprises surveyed rated challenges related to cost, budget and financing as the fourth biggest barrier to cloud.
But what is the challenge? Why are things different than they used to be? It turns out that the two biggest benefits of cloud are also the two biggest challenges: on-demand pricing and flexibility.
Let’s look back 10 years ago – how was IT procured? Well the IT department was always involved. It was the IT department’s budget that was being spent, the IT department would take the lead in choosing a solution and would ultimately deliver and manage it. The IT department controlled expenditure and usage. Could the marketing team suddenly bring up an application 10 years ago? Not usually – generally, IT were always the gatekeepers.
Now it is perfectly feasible than a CMO with no IT knowledge could sign up his or her team to a service like Salesforce without IT knowing about it. Obviously, the once-omnipotent IT department would rather this weren’t the case, but now the genie is out of the bottle. End users know they have access to cloud services and the use of a credit card, and this shadow IT is obviously worrisome to security-concerned CIOs. So now end users have the flexibility to choose the services that best suit their needs, but they also have the risk that on-demand, in-arrears pricing will let them consume without control. The result: a big bill at the end of the month, and a shock for the controller of that budget.
How big is this issue? 451 Research’s Voice of the Enterprise service previously found that 33% of IT end users weren’t confident that their cloud costs were under control. From the same sample, 25% weren’t doing any evaluation whatsoever. For big or small enterprises, these are worrying statistics. The only real option for enterprises is to embrace the cloud – give end users access to on-demand cloud services, but ensure they are managed, integrated and controlled – ‘IT as a service.’
So IT can offer its end users access to cloud services, while keeping a level of control. Everyone wins! But here, ‘control’ is the operative word. Who will pay for the cloud service? From whose budget will it be assigned? Who has access to what? These are simple questions, but the answers aren’t as obvious as one might think. The fixed processes, policies and approvals already in place for traditional procurement of IT are likely to collapse in the on-demand, pay-as-you-go, rapid deployment world of cloud.
For access to cloud resources, there must be a feedback loop of information. The controller of the company’s IT needs information on usage and spending to assess and allocate budget on a constant basis, and to control end users’ access to resources. Ultimately, the service provider is in the best position to give end users this meaningful data. Without this real-time information, CIOs will be left in the dark until the next invoice. And once that invoice arrives, the damage is done – the CIO is liable. The usage information must be turned from data into action – analyzing, processing, identifying risk, warning departments and seeking approvals must happen on an ongoing basis, before things get out of control. With the rise in cloud consumption, this requirement becomes only more critical and important.
It’s clear that usage data and reporting isn’t just a bonus – it should be of genuine concern both to end users and to the service providers who want to service them. And we still haven’t discussed expenditure optimization. Shouldn’t enterprises be able to identify when cloud resources aren’t deriving value, shouldn’t enterprises be able to detect when a cheaper resource will meet their needs, and shouldn’t they be given access to cheaper resources in times of lower demand? These are reasonable requests. Enterprises will increasingly want to make savings on cloud, and usage (and price data) will be critical to their efforts.
In the multi-cloud world, financial data really drives the benefit of different best execution venues. Usage data gives CIOs the ability to determine what workloads are driving expenditure; pricing data gives CIOs the ability to reassess if there are cheaper best execution venues in the market that meet their needs. Over time, enterprises are increasingly going to want to use a number of cloud providers to meet all their differing needs – financial reporting gives them the ability to assess which can best address these needs.
Financial reporting – a review
Amazon Web Services leads the cloud market in financial reporting. It not only provides access to usage and rate card data via a download and API respectively, it also provides a range of built-in tools to report and optimize reporting. Its Trusted Advisor can suggest opportunities for savings, its Budgets and Cost Explorer capabilities allows analysis of current and future spending, and its powerful API gives access to both consumption and rate card information. Consolidated Billing gives enterprises cross-department reporting capability.
We were very impressed with SoftLayer’s API capabilities – its API allows access to usage and rate card data, with a number of filters. It’s also possible to derive a specific quotation for a list of requirements, through a quote capability that transcends cloud services. The SoftLayer experience was seamless to begin, with our resource being spun up and a friendly sales rep calling us to discuss our usage. The API was powerful and well documented. Alas, 12 hours later our account was suspended. Two days later, after validating our email, providing EU identification and having spoken to a rep, SoftLayer informed us that further proof of our identity was required. This experience further validated our view that business processes, not technicalities, are the real challenges to cloud adoption.
We wonder if our account was flagged because we used a Gmail account to register and were unable to provide a VAT registration – after all, we used a valid credit card, a real identity and consumed just one resource, none of which seems suspicious. If this is the case, independent traders with relatively low income (below VAT thresholds) may find SoftLayer not as easy to access as the likes of AWS, Azure and Google. Those worried about data protection, especially in Europe, may also be unhappy to learn SoftLayer did not accept a UK/EU driving license (despite having asked for government ID, a requirement that a driving license meets). Some European end users may be, understandably reluctant to provide a copy of their passport to an individual located on a different continent. Of course, many would prefer that SoftLayer be over-cautious when it comes to potential fraudsters – after all, SoftLayer has to protect all its customers, and this is of particular importance when capability is multi-tenanted.
Azure provides a GUI for financial reporting through Resource Manager, and an API to access usage data. The rate card and usage APIs are in preview, and provide access to pricing data. Authentication and permissions were not easy to configure, with the use of a service principal being required to gain access.
Google provides usage information currently through Billing Export into a storage bucket, and a JSON pricing list is available at https://cloudpricingcalculator.appspot.com/static/data/pricelist.json. Currently, Google’s Cloud Platform’s console provides billing reporting. We expect Google to enable usage information via the API in the future.
Reporting stood out as a weakness in Rackspace’s capability. Even though Rackspace is keen to differentiate itself from the hyperscalers, even enterprises using value-adding managed services will need this capability. Rackspace should develop this capability sooner rather than later, especially now it is managed cloud services located on third-party clouds such as AWS. Dimension Data, CenturyLink and Verizon provide cost billing data via API and GUI, but do not provide a rate card API. CenturyLink also allows users to change the prices of sub accounts to reflect the value added by the IT department in an internal broker model.
Summary
In this analysis, we provide a review of API report capability from a number of cloud providers. We examine this in two ways:
- Usage report – access to report on what resources were consumed, by whom, and what are the cost implications.
- Rate card – access to report on what services are available and their list prices.
Cloud financial reporting by company
|
Source: 451 Research
Third-party reporting tools
A range of third-party reporting tools rely on these APIs to derive usage information. The likes of Cloud Cruiser, Cloudyn and Cloudability use APIs to extract consumption data from AWS, Google and Azure. For providers without usage-reporting APIs, resource-deployed agents can measure consumption independently and report to the platform. In the private cloud space, Ceilometer is a resource usage monitor for OpenStack, but many regard it as a work in progress. VMware provides financial reporting through vRealize. The benefit of an agent is that it provides an independent measure of consumption, separate from providers’ view of the world.
Many broader cloud management firms have added cost management technology, including CloudHealth, Ensim, Scalr, RightScale, CloudCheckr, RISC Networks and Apptio. These companies offer cost management alongside other capabilities or advanced financial management. Apptio in particular is a market leader in broader IT estate financial management. The enterprise-play cost management market has consolidated, with Newvem Insight, CloudVertical, CloudRows and Sensible Cloud all disappearing from the scene – the reigning kings of pure enterprise cost management are now Cloudyn and Cloudability. Cloud Cruiser also stands out as a powerful player in this space, but we see it as having a broader capability than the enterprise focus of Cloudyn and Cloudability. As service providers own financial reporting capability improves, third-party tools will have to improve multi-cloud support and analytics to stay ahead of the game.
ClusterK, acquired by AWS in 2015, and Spotinst take this capability further, purchasing spot instances at lower prices than on-demand on behalf of end users, while managing availability.
