Blog

Why Is It So Tough to Make a Useful Cloud Abstraction?

By Atos Apprenda Support

clouds and rocks

Here at Apprenda, we’re proud of our ability to run on any set of OS images you give us. Whether they’re on top of bare metal, hypervisors, or cloud, we can run it.

We think open infrastructure support is an important way to facilitate an enterprise’s IT-as-a-service vision. We also believe that the intricacies of managing infrastructure (or even knowing where an application is running) should be hidden from the consumers of a service.

As the enterprise cloud vision matures and the platform becomes the focal point, we’ve begun to get more requests to make it easier to add capacity to the resource pool underneath Apprenda. Sticking with our mantra of hiding the infrastructure details, we’ve been exploring how to interact with the wide variety of platforms used by our customers in a way that isolates them from the differences in management control.

Ideally, we’d like to be isolated from those differences as well. Managing infrastructure will probably never be a competitive advantage, and we’d rather focus our time on delivering the best possible platform to meet the agility, security, and governance needs of the modern enterprise.

Unfortunately, abstracting away the underlying provider is much easier said than done. While something like Chef or Puppet can help, it’s hard to take a dependency on a full infrastructure-management system.

There are a few third-party cloud abstraction layers like Apache jclouds or Dasein Cloud that attempt the task. Ideally, they should hide all the complexities of cloud-provider interaction behind a consistent facade. But the abstraction layers tend to be leaky or require many capability checks.

Let’s take a look at the Apache jclouds example for creating a Windows machine on EC2. (Note: I removed some of the boilerplate and error-handling code.)

jclouds_ec2_windows_2

The node creation steps (lines 18-29) are nicely abstracted. There is no explicit knowledge that we’re deploying to Amazon EC2. Everything environment-specific comes from the passed-in arguments.

But look at how we get the credentials back (lines 57-59). We have to explicitly reference the AWSEC2Client (see line 11) to access the Amazon Windows services.

What if I wanted to build a Windows VM on Azure? I can’t use the exact same code because of the AWS dependencies. I either need to branch based on the targeted provider, or build an abstraction on top of my abstraction service.

Dasein Cloud takes a different approach. It has a much more robust abstraction that allows each of the various cloud adapters to advertise which services it provides. This allows Dasein to avoid the common abstraction problem of building to the least common denominator, but this also leads to some potentially very ugly code. (Again, I’ve removed error handling and other boilerplate code for clarity.)

dasein_cloud_vm

You’ll first notice there is no reference to a specific cloud provider. Instead you get all sorts of checks to see whether the chosen provider requires a given approach (line 35, for example), and a lot of object matching (e.g. line 26).

It’s a much tighter abstraction, but the developer experience is poor. Sure, we can hide the complexity within code like this sample, and not bother the ultimate consumer with it. But it’s still a pain to deal with.

Despite using code samples from the abstraction frameworks to demonstrate the issues, I don’t blame the designers of these frameworks for the outcome. This problem stems from the providers themselves.

The desired need for a consistent API across infrastructure providers is a well-covered topic. My intent is to approach this, not from the “marketitect’s perspective,” but from the implementer’s. What I’d like to see, and what would truly enable hybrid-cloud platform scenarios, is a consistent way to set up the basic components of a virtual machine across all providers. One that includes things like setting and retrieving credentials, setting the NICs, etc. without jumping through the hoops that the cloud abstraction libraries need to do.

While it will likely take some time to get there, basic compute resources will become simple commodities. The major IaaS providers are all innovating around higher level services and efforts like OpenStack are working to introduce compatibility across providers (the success of that mission is uncertain, but the mission itself is commendable). But even as we move closer to unification across cloud providers, we’ve still got to deal with the traditional virtualization vendors that make up a large part of the internal data center footprint. And I don’t see any incentive for those vendors to make it easier to tie their components into a cohesive hybrid ecosystem.

Until we can achieve this level of compatibility, anyone who wants to support a hybrid cloud faces the difficult choice of whether to let cloud details leak out of their abstraction layer, or to compromise the user experience with innumerable safety checks. We’re more than willing to take on this difficult task and to enable our customers to have a clean hybrid cloud approach.

Atos Apprenda Support