The “Open” Cloud is Coming

Posted on by su


The rhetoric from open source pundits about the ‘closed’ cloud has already started.  From Richard Stallman to others there has been nay-saying related to the cloud being ultimately closed and proprietary.  This is a fundamentally misguided understanding.

Where ever you stand on ‘free markets’, there is no doubt that market forces play a major role in technology adoption.  In this case it means that not only can cloud computing be open.  It will be open because of market pressures.

Open Source and Market Forces
Without a doubt open source has benefited from market forces, becoming widely embraced.  For many (e.g. startups) the cost economics were too compelling.  Others (e.g. Google) reaped both the benefits of cost economics and the openness of the source code itself.  Certainly even those companies who embraced open source for the ability to see the source code first and foremost were looking at it with an eye towards saving money elsewhere, not from ideology.

Market forces that drive open source are the same as those that drove the opening of Internet walled gardens.

Walled Gardens, Market Forces, & Democracy
The openness of the Internet is certainly not in doubt.  Despite ongoing attempts by providers to profit by selling a better quality of service to those with bigger pocket books, Net Neutrality advocates regularly stymie their efforts[1].  When large publishing sites and newspapers have closed off their access they are largely scorned by the individual Internet user.  Back in the day, we used to paraphrase John Gilmore saying: ‘the Internet routes around censorship, problems, or politics.’  Meaning that, like the network protocol (TCP/IP) upon which it is based, the Internet is largely immune to efforts to disrupt it.

Why is this and what are market forces really?  ‘Market forces’ are the effect of people voting with their wallets.  People want open, cheap, and accessible on the Internet.  It’s been proven over and over countless times now.  Internet-based walled gardens will eventually fail with few exceptions.[2]

Cloud computing is no exception.  It’s got to be open, cheap, and accessible.

Opening of the Cloud
It’s very easy to see the current advent of cloud computing as being closed.  It is.  Historically most new Internet markets and technologies are closed until they mature.  The cycle can be described as:

  1. A new Internet market/technology starts as closed and proprietary
  2. Under pressure in a high growth market it begins to open
  3. When market or technology matures many of the technologies are ‘open’ or ‘open enough’

Consider for a moment the now standard x86 processor instruction set, the SQL relational database interface, or the availability of APIs for just about every Google service.

Already there are the inklings of people working on concrete standards including ones for cloud computing infrastructure (Open Cloud Computing Interface (OCCI) a working group of the Open Grid Foundation (OGF)) and the portability of virtual machines (Open Virtualization Format (OVF)).  Once these standards are finished and embraced by even a few clouds many customers will opt to choose cloud providers that use them, avoiding vendor lock-in and give choice to customers.

Clouds are not Software
Being ‘open’ means something different for Internet-based services, such as cloud computing, than it does for software.  The essence of cloud computing is that you are renting resources from someone else.  This is powerful and provides tremendous advantages to the customer.  It’s also no different now than it was when it was calling ‘hosting‘.  The very nature of this kind of offering is that customers won’t have 100% control.  Someone has to run and maintain the system that you are renting in a professional fashion and retains control … up to a point.

Open Cloud Services?
Where are the points of ‘openness’ for Platform-as-a-Service (PaaS) and Infrastructure-as-a-Service (IaaS) businesses?  It’s fairly simple to assess this.  What is ‘open enough’ to allow you to take your application and move it from one PaaS or IaaS provider to another?  Some obvious points come to mind:

  • Data & code portability
  • Widely available programming languages and libraries
  • Interoperability through widely embraced and supported APIs
  • You ‘own’ it and can take it with you

Where ‘it‘ is your web application (PaaS), virtual server (IaaS), and data.  Vendor lock-in comes from disallowing you to leave and take your data, tools, or software with you.

Platform Lock-in
If we were to measure PaaS providers like Salesforce.com’s Force.com, Google App Engine (GAE), and Heroku they might be graded as follows:

  • Force.com:  ‘D-’
  • GAE: ‘C’
  • Heroku: ‘A’

Force.com creates extreme lock-in via its closed proprietary language Apex that is unavailable outside their service.  GAE a bit less so as it supports widely available languages such as Java and Python.  On the flip side, in order to leverage GAE in a meaningful manner you must use Google proprietary libraries and technology to access BigTable instead of a traditional database.  Heroku comes out smelling like roses allowing you to use a widely available language like Ruby with little or no modifications.

Infrastructure Lock-in
Unlike PaaS providers, infrastructure services do not dictate languages or libraries, meaning that there is even less areas for them to lock you in.  By providing a basic building block of a ‘virtual machine image’ IaaS providers allow customers to ‘own’ and ‘control’ their servers.  Even now, the 800lb gorilla of cloud computing, Amazon Web Services, allows you to download their Amazon Machine Images (AMI).  You can download one now and run it on your own infrastructure.  You can take it with you.

The primary way in which infrastructure providers create lock-in is twofold.  First  by using a proprietary API that is not widely adopted and second by disallowing portability of virtual machine images.

Once tools are written for a closed API you’ll get little or no reuse of those tools.  This is why folks like GoGrid, my employer, and our competitors are actively working together to create an open API for controlling IaaS systems.  As part of that discussion we’re also actively working on supporting standards like OVF that allow virtual machine image portability across hypervisors and clouds.[3]

If you were to grade a bunch of us today, however, it wouldn’t be bad, but it could be better:

  • VMware vCloud/vSphere: ‘C-’
  • GoGrid: ‘C-’
  • Amazon: ‘C-’

You might score these differently, but for me VMware gets marks for widely adopted APIs, but demerits for dragging their feet on standards they helped create like OVF and being missing from conversations like the OCCI effort.  GoGrid has an open license on our API, but doesn’t allow downloading or uploading virtual machine images [4].  Amazon has widely adopted APIs, but under ambiguous licenses and quite a few proprietary technologies (e.g. S3, SimpleDB, and SQS) that encourage lock-in.

The good news is that standards for cloud interoperability (OCCI) and virtual machine portability (OVF) are either here or imminent and many providers are planning to adopt them.

Summing Up ‘Open Services’
It means something different to be an ‘Open Service’.  We all want to be able to export and move our applications from Google App Engine or Force.com when we leave.  We also want to take the virtual machine images that we build on Amazon or GoGrid and move them elsewhere.  Today we  can, but it’s just a nuisance.  New standards will make interoperability and portability easy and convenient. If anything, cloud computing is becoming ‘open’ faster than anything before it.

Ignore the naysayers.  Customers want choice and they will have it.  Choice is driven by open standards, cheap resources, and easy ‘self-service’ access.

The ‘market’ demands it. 

 


[1]  Yes the battle isn’t over, but there is clear and unrelenting demand for Net Neutrality.
[2]  If in doubt compare the relative approaches of MySpace and Facebook.  Facebook, in opening it’s platform, was able to make a come-from-behind ‘win’ because of this.  It’s uncommon that the #2 player in a given market/space will blow past the #1 so quickly.  It’s largely due to Facebook opening their platform.
[3] The differences between virtual machines running on different hypervisors is much smaller than people think.
[4] It’s on the roadmap.

This entry was posted in Cloud Computing, Technology and tagged , , , , . Bookmark the permalink.

Previous post:
Next post:

  • http://www.jordanrinke.com JordanRinke

    OpenNebula is an open source cloud engine, worthy of mention even though it is geared at the “private cloud” currently and your blog article deals with the cloud on a much broader level. The Cloud Plug-in feature is of particular interest in relation to how open the cloud is / will become.

    http://www.opennebula.org/doku.php

  • http://neotactics.com randybias

    Thanks OpenNebula is good stuff. They are also a part of the OCCI OGF working group efforts to standardize the API for cloud infrastructure!

  • http://somic.org Dmitriy

    “Customers want choice and they will have it.”

    Not necessarily.

    Example: Do you see many pink cars in the US? (Don't know about other countries). No. Do you want to be able to pick any color you want for a car? Sure, I assume most people would love to be able to select some bizarre colors, or at least they won't mind having such possibility.

    The reason we don't see many bizarre colors is because even though customers may want it in theory, too few are willing to pay premium for it. So vendors cant justify this feature, and so they don't offer it in all but high-end markets.

    Please note I am not saying cloud computing customers will never have choice. All I am saying just because customers want something doesn't mean they will always get it.

    My thoughts on cloud computing interop (somewhat related to openness) are at http://somic.org/2009/04/04/incentives-and-cloud-computing-interoperability/.

    Cheers,
    Dmitriy

  • Pingback: Rational Survivability » Interesting Nuggets: Quick Tidbits I Find Compelling

    • admin

      Fair enough. ‘open enough’ is still ‘open’ as far as I’m concerned. I know some folks will disagree with me on this, but when I say ‘widely available’ that’s what I mean. De facto standards are usually just as good as open standards in my book.

  • http://neotactics.com randybias

    Dmitriy,

    Thanks for your comment. I'm afraid I have to disagree. There may be some folks who want pink cars, but almost clearly a minority. If they were big enough they would constitute a real 'market force' and car manufacturers would produce pink cars. More importantly, cars and the cloud couldn't be further apart so your analogy is a bit off-base.

    The cost to a car manufacturer to provide, ship, distribute, and stock cars with infrequently requested colors for small groups is prohibitive. The Internet and Internet-related services like the cloud, by their nature allow for smaller markets (in aggregate) to be worth servicing once a cost-benefit analysis is done. This is the whole 'long tail' in action. In effect, there can be a large 'market force' because of the reach available to Internet services.

    In terms of choice, the main thrust of this article is that customers will (and already do) demand choice. GoGrid already sees a number of customers move from other cloud providers who don't provide them with the kinds of choices they want and vice versa. The 'market' usually votes with it's feet.

    My assertion is that once a segment of the cloud computing market adopts 'open', 'open enough', or 'de facto' standards customers will begin to move to those segments away from the more closed segments. Just like they have with social media sites, news, and related. People want cheap, easy, and accessible.

    –Randy

  • http://blog.heroku.com jamesheroku

    A great read, we couldn't agree more. And thanks for the good grade, we put a lot of focus on openness in our architectural decisions.

    Happy to talk more about “openness” any time.

  • Pingback: Twitted by GirishB

  • Pingback: blog.dsa-research.org » Archives » Interfaces for Private and Public Cloud Computing

  • Pingback: Cloud Futures Pt. 3: Focused Clouds | Cloudscaling