VMware driving container coopetition

Is software really eating the world? Certainly a revolutionary conjuncture of trends would suggest this is so. While new digital models are displacing traditional brick and mortar businesses across the industry spectrum – Uber and Airbnb are only the most celebrated examples – the app explosion is delivering an unimagined diversity of novel services to citizens, customers, businesses and their partners, who in turn are using modern applications to unlock new market opportunity. But software is also rocking the IT industry, a point that was driven home at VMworld 2015 as the veteran data centre disruptor announced a slew of innovations aimed at providing support for the developer community – and the IT operations folks tasked with delivering their product in production mode. For VMware, software is both the vehicle for customers’ digital transformation and the means to shift manual provisioning processes in the data centre into autodrive to optimize IT service delivery of new applications.

To borrow from Churchill’s lexicon, VMware’s vision for the app-intensive future outlined in key VMworld announcements was about cloud native apps, wrapped in containers, wrapped in virtual machines. Specifically, VMware’s new approach to helping customers support the deployment of modern applications was launched at the event through two new technology solutions: vSphere Integrated Containers and the Photon Platform, which both aim at enabling customer organizations to take advantage of container technologies in modern app development without compromising the IT department’s requirement to ensure enterprise grade security, performance and resilience in service delivery.

In the IT world, containers have become the poster child for rapid, agile development. They perform a number of functions: containers enclose a program, code or micro service in software that connects directly to the OS to speed app deployment and helps move applications across different infrastructures, including the programmer’s laptop, test bed and cloud production environments. Since they share the host OS (ex. Linux), they consume few resources, offering what many observers have suggested is a lightweight, portable alternative to virtual machines. As a barrier between application code and the infrastructure, they help to identify responsibility for performance issues, assigning this to developers or IT to improve resolution times. In some cases, container environments even serve as PaaS through the collection of like applications that can be used as building blocks in the creation of new applications. While containers first appeared in early 2000 (ex. in Sun Microsystems’ Solaris Zones), as open source, Linux-based container technologies like Docker have been hardened, standardized and certified for production deployment, their reach has extended beyond web or mobile apps, garnering greater interest in the enterprise space, and with it, the attention of many of the larger vendors. Docker, for example, has relationships with providers ranging from Google, to Red Hat to Microsoft – and now VMware.

Container capabilities have led a number of industry watchers to question the long term viability of virtual machine based platforms. If containers move more quickly, relieve the developer of writing to the underlying system and consumer fewer OS resources, a recurring question has been ‘what purpose does the VM serve’? VMware has an answer for this, showcased in innovation initiatives such as Project Bonneville, Project Photon and VMware’s Instant Clone technology that have been designed to integrate containers into VMware virtualization, while providing enterprise capabilities that are unique to the vSphere platform. As EVP and GM for VMware’s software defined data center division Raghu Raghuram noted at the event, “we hadn’t see the integration of new cloud apps yet in the marketplace, so we worked to solve for this problem. We see our role in the stack as being very complimentary; we are looking to deliver an industrial strength solution.”

So what benefits does this integration provide, and how has innovation in VMware’s Cloud Native App business unit managed to simultaneously serve the interest of the developer community in speed and agility and the IT department’s need for security and manageability?

At a basic level, the VMware approach is agnostic from a container technology standpoint. Instead of requiring the user to stick with a single platform in a shared OS model, the VMware vSphere Integrated Containers solution provides a common infrastructure that enables IT teams to support any application, including containerized applications, which may take advantage of multiple container ecosystem solutions such as the Linux CoreOS, Tectonic, Docker, Google’s Kubernetes, Mesosphere’s Data Center Operating System or Pivotal’s Cloud Foundry. This provides developers with a level of flexibility that is not available through use of a single container platform.

But by wrapping the container in a VM, the new technology allows users to take advantage of existing investments in VMware infrastructure, processes and staff resources to deliver unified management of both modern and legacy applications, whether these live on premise, in a provider’s cloud or a combination of the two.

Over the past several years, VMware has enhanced security in its NSX platform, introducing micro segmentation and the embed of security protocols at the VM level, capabilities which can now be extended to the container. Jared Rosoff, sr. director of product management and architecture, Cloud Native Apps at VMware, described how this is achieved. In non-VMware environments, multiple containers typically live on a guest OS and with a container engine make up a virtual machine. With vSphere Integrated Containers, this has been broken up, the Linux OS and container engine removed so “you literally have containers residing straight on the hypervisor, and each container is a virtual machine.” A new VM for each container is instantly created using the Instant Clone feature of VMware vSphere 6.

A primary advantage of this lies in the isolation that is created for the virtual machine and with it the container within. As Rosoff explained, security benefits are key: “we know the security properties of the hypervisor: it’s very hard to break out of a hypervisor, out of VM and into another VM host. But if you are able to compromise one of these containers, with OS virtualization, it’s relatively easy to compromise the rest of the containers running inside the host as well. When the virtualization boundaries are around the containers themselves, even if you were able to compromise one, there’s no way to break out of it and compromise the others running on the host.”

The one container per VM approach also offers significant management benefits. By providing the admin with visibility into the container, which was missing in other types of deployments (containers appear as VMs in vSphere, are identified by the developer with meta data and other techniques, and all have logical names), Rosoff noted, the solution allows the administrator to manage containers just like other VM assets. “Since containers are now VMs, the entire VMware ecosystem of management software that understands vCentre and hypervisors and guests can manage these,” he added. For example, though NSX, containers can be integrated with the rest of the data centre, and hence connected other security provisions such as quarantine, forensics and/or network monitoring for troubleshooting.

Integration with the VMware virtualization platform enables the container user to leverage other platform capabilities, including: vSphere storage (Virtual SAN and vSphere Virtual Volumes) to enable the provisioning of persistent data volumes – support for containers through delivery of cloud-native databases; the vSphere Distributed Resource Scheduler, vSphere High Availability and vSphere vMotion solutions to reduce downtime and assure SLAs for container workloads; vCenter Server to view and manage containers without additional tools and training; as well as the vRealize Suite to provide consistent management and configuration compliance across both private and public clouds. In other words, through integration, the VMware suite manages the ‘heavy lifting’ in terms of security, storage provisioning, and cloud management that is native to the VM, and extends this to the containerized application environment.

But what of weight? Here’s where the Photon Platform kicks in, with a much lighter OS, and, as VP & CTO, Cloud Native Apps at VMware Kit Colbert described it “just enough virtualization” to deliver the speed and agility associated with containers along with the security and management provided by VMs.” Because of the lightweight Photon OS (the disk footprint is 25 MB; and Instant Clone takes less than 1 second to create a VM), the one container per VM approach is also highly efficient. Colbert calls Photon the “evolution of virtualization” – a platform that offers speed as well as management, and which is invisible to (developer) clients.

The Photon Platform consists of three basic elements: Photon Machines, including a new ESX Microvisor based on VMware’s ESXi (the hypervisor); a lightweight Linux operating system for containers that has been optimized for the VMware environment; and the Photon Controller, a multi-tenant, API-driven control plane that has been optimized for scale, churn and high availability. According to the company, the controller can speed the creation of thousands of new containers per minute, and support hundreds of thousands of simultaneous workloads. It also incorporates Project Lightwave identity and access management which enables single sign-on, authentication, authorization and certificate authorization across the application stack and infrastructure to deliver security, governance and compliance to container workloads in cloud-native applications. Many of the Photon components will be shared as open source, and the platform will support dynamic continuous integration, PaaS or SaaS deployments, as well as data analytics clusters running Hadoop or Spark. In future, VMware is looking towards Photon integration with NSX, vSAN and the Realize Suite to layer on the benefits of VMware’s standard virtualization platform.

According to Colbert, the best use case for Photon is the Greenfield data centre and applications – for DevOps teams that need to build out large pools of commodity computing capacity that would run cloud-native applications, for example, or for SaaS organizations that would run cloud-native applications at scale. The vSphere Integrated Containers solution, on the other hand, is targeted more at organizations with existing, and heterogeneous environments with legacy apps that need to be managed alongside more modern, container-based applications. VMware’s ability to offer solutions across both customer environments, which build bridges between legacy and modern environments, and marry experimental activity with production requirements without disturbing the developer’s workflow is a clear differentiator for the company. Asked at the event if VMware’s is “behind the innovation represented by container movement,” Colbert’s answer was clear: most customers remain challenged by the operationalization of next generation technologies, and VMware is in fact positioned “at the inflection point” where they can help customers who are just now embarking on the container journey.

Going forward, VMware and Pivotal, a PaaS VMware spin off, have plans to collaborate on bringing joint solutions to market; a joint offering that combines VMware Photon Platform and the Pivotal Cloud Foundry solution to accelerate the deployment, management and integration of cloud-native application development into the production stack was announced at VMworld 2015. In addition, Project Lightwave is available now as an open source project on GitHub as is the Project Photon OS (GitHub). VMware plans to make the Photon Controller available as a private beta in Q4 2015.

For those looking to learn more, VMware has prepared an introductory “Field Guide to Cloud-Native Applications.”