Open, Cloud, Confusion

Posted on by Randy Bias


This weeks’ re-launch of Cloudscaling was amazing. It was all we could have expected and more. My only regret was not being able to walk the halls at Cloud Connect as much as I would have liked, but I think I made up for that during our reception, which was a blowout. A weeks’ worth of hallway conversations in only 3-hours time.

That being said, no launch goes flawlessly. We had some misfires early on and some controversy right after the new website launch. I have a well deserved reputation for being controversial in the cloudosphere so it would probably be a very disappointing launch if I didn’t cause at least a *little* stir. :)

I thought it made sense to address some of these questions and concerns, while also owning up to any of the criticisms I felt were warranted.

Open + Cloud
We took a little heat from Chris Hoff, a friend a fellow ‘clouderati’, about so-called ‘openwashing.’ I think some of Chris’ criticisms are fair[1], but mostly I think his concerns highlight a failure to communicate on our part. This is the great thing about public forums and market feedback. It makes you a better company and frankly those that are sincere and authentic can thrive in that environment. I believe that’s Cloudscaling and so I would like to explain briefly the disconnect.

Chris’ argument boils down to a perceived overloading of ‘open’ as meaning ‘scalable’, such that my litmus tests for ‘open cloud’ are bogus. Except, we aren’t claiming that open is scalable. We’re claiming that ‘open cloud’ is scalable. It’s not open applied to cloud. It’s open AND cloud, where by cloud we mean scalable infrastructure a la AWS.

As long time readers know, I’ve always been uncomfortable with the term ‘enterprise cloud’. Two years ago I was crystal clear about my thinking and haven’t changed my position since. The basic gist is this: so-called ‘enterprise clouds’ aren’t really clouds, they are virtualization 2.0 cloudwashed til it glows. At Cloud Connect 2011 (last year), my keynote reiterated that enterprise clouds are a myth.

Hey, give me points for consistency!

Here’s the problem: everyone with an ‘enterprise cloud’ believes it’s a cloud and their vendors tell them it’s a cloud[2]. It’s hard to throw this term away as much as I would like to, but in this case I need to use the term everyone else does in order to get my point across. In this way it reminds me of the co-opting of ‘hacker’ in the general populace. All of the old skool UNIX folks were unhappy with it, trying to push ‘cracker’ instead, but generally failed. ‘Hacker’ is now synonymous with black hat[3] types. Enterprise clouds will ultimately be synonymous with virtualization 2.0, not clouds in the long run.

So, here’s where we failed. We didn’t make it clear that we are talking about open (freely available, no-vendor-lock-in, preferably open-licensed) PLUS cloud (elastic, scalable, secure, robust). This is our fault. The challenge being, of course, it’s difficult to have any kind of nuanced discussion while creating bite-sized marketing that is easily consumed and tastes great.

No problem. We’ll work on making that part more clear, but hopefully you can see that there is no disconnect between what we’ve said all along, how we are positioning, our marketing message, or are choice to push for a greater understanding of what open clouds are.

Open or Open First as a Strategy
For some, our announcement is seen as a way to justify their early positioning. That’s fine. We think that many people can build open clouds and open cloud products. It’s an umbrella we welcome others to be a part of, even non-OpenStack vendors and solutions companies.

For others, it’s easy to see Cloudscaling ‘jumping on the open bandwagon’. It’s really not for me to say if we are ‘openwashing’ and I’m not going to claim to have been ‘open first’ (does it really matter??). The marketplace will determine that and folks will get their pound of flesh as they see fit. From where I am standing we have been consistent and true.

We are committed to open-ness, open source and open projects. Both my credentials and Cloudscaling’s predate our announcement by a long time. Here’s a taste:

More importantly, we see being open as a legitimate business strategy. Perhaps Dave Asprey’s analysis at the TrendMicro cloudsecurity blog sums it up best. We are pushing open cloud because we think it matters AND we’re open because we think it’s the right business move.

Understanding Production Open Clouds
Cloudscaling’s engineering and architecture teams includes myself, key back-end engineers from GoGrid and other large scale systems (e.g. eBay). We also lean heavily on Chris Brown (the original lead developer for Amazon’s EC2) and Ben Black (one of the core AMZN/AWS network architects) who are both on our technical advisory board. Cloudscalers know a few things about building production-grade cloud services.

All of the above is public information. The following is internal Cloudscaling lore that you’ll have to take my word on.

In early 2009 my engineering team lobbied hard to build our own IaaS software stack, just like OpenStack. They had the experience and DNA to do this. It would be go number 3 or 4 for some of them. The problem was that there was a long list of commercial and open source entities out there already with IaaS stacks. We were a young business and I didn’t want to be IaaS stack #26. The other major problem is that we knew what it takes to build a production cloud service and allocating a VM to a hypervisor was the trivial part. The much harder problems are in operations and ongoing management and still under appreciated area.

So, I resisted and believe me, people were unhappy. Worse yet, I asked them to use one of the existing commercial stacks, now open source, that was poorly architected Java and just about had a revolt.

When I found out about OpenStack in mid-2010, I was relieved. This was what we needed. An open source project with real momentum creating a greater community we could be a part of. I didn’t want to own the creation of the IaaS stack, I just wanted to participate and make someone else’s stack better. That was when the first parts of our generally open strategy started to form. That was also when we knew our future course was software + services. It didn’t hurt that Cloudscaling had a bunch of Pythonistas and that OpenStack was built around Python.  We also felt that the core Nova and Swift architectures were sound.

Summing It Up
We don’t claim to be the most open company out there. Openness is a spectrum and we want to be as open as possible. Other than what we are doing already (see below), we will continue to learn and evolve. We plan to monetize our business outside of the core open components of OCS. We are going to give back network architecture blueprints and documentation, hardware blueprints, open source software, and a raft of other ‘trade secrets’ we have kept close for a long time. I want everyone to build awesome open and scalable clouds. I expect that Cloudscaling will be one of the best and we’re fine competing with the other great teams out there.

The truth however is that we have been interested and committed to open for a long time because our approach is a full-stack approach and we can’t build it all. The only way to succeed is to leverage what exists, which is other open projects. What we plan to monetize is outside of the core open components so there is no conflict. We aren’t open to be cool. We’re open because we care, we want to make great products, and we want to compete effectively against folks in the marketplace. Just like everyone else.

So, feel free to criticize, and we welcome the feedback, but I prefer you do it in context.  Disagree with my definition of cloud if you like.  Meanwhile, we’ll get to work fixing some of our marketing content.

No harm, no foul.


[1] They also deserve to be addressed en masse; however, for this article, I’m choosing to stay focused on the core issue: “open”
[2] Not only this, but honestly, I’ll probably just as criticised for walking around slinging the ‘truth’ in our marketing material that enterprise clouds aren’t ‘true cloud’; damned if you do and damned if you don’t.
[3] Sorry, I still can’t bring myself to use the term ‘hacker’ since it’s been co-opted and even the Wikipedia page is a travesty in that it’s taken on the popular term. C’est la vie.
[4] Amongst a number of others notable production OpenStack deployments, both storage and compute, some of which I can’t talk about unfortunately, but the Intertubes will mostly tell you if you search for it.

This entry was posted in Cloud Computing. Bookmark the permalink.

Previous post:
Next post:

  • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

    Hi Randy,

    I have few questions lingering in my mind:

    - Isn’t committing full-on to OpenStack is a lock-in in itself? Granted, you can swap compute or network or storage boxes. But you can do it with other stacks, too. And have commercial support available in many cases.

    - You’re saying that you are releasing whole bunch of various blueprints; but haven’t other players done so, too? There are a bunch of published “pick up and go” style reference architectures available, which are not based on OpenStack. APIs? Others document and publish theirs openly, too. Does that not make them “Open”?

    - Availability of professional quality training, certification, and up-to-date documentation is a keystone of successful technology business. How are these things addressed around OpenStack?

    I’m also not convinced that the availability of the source code
    automatically makes product somehow more attractive to everyone. To
    those with highly skilled workforce in house? Yes. For those without
    such? Likely no. I may be wrong, but I suspect there are many more companies of the second kind out there.

    Cheers,

    – Dmitri

    • http://cloudscaling.com/blog randybias

      Hi Dmitri, apologies for missing this the first time!  Mea culpa.  Thanks for your patience. Your answers following.

      1) OpenStack is open source software, which reduces lock-in by giving you the option of taking on the software development and engineering work for that project at any time.  Most of these stacks give you options to replace hardware boxes, but not all boxes are equivalent nor are all architectures, so that is limited regardless of whether you use open source or commercial software.  Finally, OpenStack has a large number of commercial support options.

      2) Yes, those are all open blueprints and reference architectures. We didn’t make claims otherwise.  As I’ve said elsewhere, ‘open cloud’ is not intended to be ours.  It’s a general approach to building clouds in the AWS style using open components.  The anti-enterprise-cloud, if you will.  Lots of folks will build lots of open clouds, hopefully.  We will have one solution of many called Open Cloud System (OCS), which is our answer in this arena.  OCS will have a bunch of open blueprints for hardware, networking, and related that can be used independently of OCS if people choose, but are otherwise designed within the OCS context.

      3) I think it’s too early for this so we aren’t seeing any training or certification for OpenStack but I am sure we will soon enough.

      4) You are right.  This is why Cloudscaling’s is offering a solution, not just open source software.  OCS includes our open source software (Open Cloud OS), blueprints, block-based architecture, and is deployed in conjunction with CloudPath, where we can help to build, operate, and transfer.  So it’s a complete package, not just open source.

      I hope this answer your questions.

      Regards,

      –Randy

      • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

         Randy,

        Thank you for taking time to answer. Just to make sure I’m understanding it right:

        1) “Lock-in” is then defined as inability to take control into your own hands of something you’ve invested time and money into. If so, the lack of such lock-in does not necessarily mean an ability to replace the platform if you don’t like it anymore or if something better becomes available, without causing major disruption to your business.

        2) What I guess I was asking is “how the openly available blueprints for “Enterprise” clouds are different from the openness of OCS?” I get it that OCS blueprints include hardware and networking, but I’m not convinced that this adds to overall “openness” of the solution – you’re now “locked in” into hardware that is developed to openly available specs (BTW, how does this differ from the existing “proprietary” hardware, which is developed to support open standards from IEEE/IETF/others? Cost, perhaps?)

        3) Point taken.

        4) Makes sense; so looks like for those without skills in-house the availability of source code and blueprints probably won’t be a major selling point.

        Cheers,

        – Dmitri

        • http://cloudscaling.com/blog randybias

          Dmitri,

          A quick response before I take a 3-day weekend and unplug for a bit.

          1) “Vendor lock-in” refers to your ability to move from one vendor to another.  Generally it refers to proprietary features or technology that are not replicated elsewhere.  This is how we mean it.  Open systems prevent vendor lock-in because there are no proprietary features or technology by definition.  You can add a feature yourself, you can hire someone to add it for you.  You basically have control.  Granted, smaller businesses are not in a position to leverage this, but they also get more control in that they can get commercial support or go direct to the open source community for help and support (assuming it’s a robust community, of course).

          2) The blueprints accompanying OCS are different in two key ways.  First, they specify a particular ratio of hardware (disk:ram:CPU:network) that is understood by the scheduling & network system.  These ratios are based on oversubscription models we have that help mimic the performance profile of a given system (AWS or RAX cloud).  So, for a given CloudBlock there is a hardware blueprint that is designed for AWS standard instance sizes (m1.small, m1.large, and m1.xlarge).  These blueprints don’t specify a particular hardware vendor, so they are ‘open’ in that any hardware vendor could provide hardware that matches the blueprint.  Second, the blueprints will be published as original source materials under an open license.  I don’t know that this makes the system more ‘open’, but I hope so.  The intention is to push these back towards Open Compute Project (OCP) and possibly ODCA if they will accept them.  That way if a customer wants Open Cloud OS, *without* Cloudscaling, they have the option to have a hardware vendor build hardware to the open specs that are available and they can take Open Cloud OS and marry it to that hardware, relatively easily.  In other words, they can put the pieces of OCS together themselves without us.  So, less lock-in and dependence on us or a particular hardware vendor.  That’s the intention.

          2b) Everything we are publishing is more of a generic profile for an x86 server with specific ratios, racking & stacking standards, cabling & labeling, networking configurations, IPMI configurations, BIOS configurations, and such.  These all require generic x86 servers.  The configurations won’t work with a more proprietary vendor who, for example, may not expose IPMI or uses a custom BIOS.  They will only work with commodity x86 servers that have no special ‘enterprise value-add’.  Our experience is that many enterprise vendors have non-standard IPMI extensions that are incompatible with open source software that controls IPMI.  Or have undocumented BIOS so you can’t use some of the tools out there for automated BIOS configuration.  Some enterprise vendors, like Dell, do have some stripped down configurations that should work.  For Dell, it’s the PowerEdge C series which is more a of standard x86 server without ‘secret sauce’ added.

          3) –

          4) I think it’s unclear what a smaller shop would be doing building their own infrastructure out.  Wouldn’t they use a public cloud or a private cloud run by a public provider?  As far as I can tell, if you are running your own infrastructure these days you have sufficient size to take on owning open source and/or updating hardware blueprints if necessary.  Regardless, one of the points of ‘open’ is to foment community participation, so that in many cases smaller businesses, or at least mid-tier businesses, can leverage the work of others like them.

          Have a great weekend.

          –Randy

          • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

            Randy,

            Thanks for the detailed answer, this helped quite a bit.

            On a slightly different topic – have you written anything around major types of IaaS needs of mid-market Enterprises (other than the likes of Netflix), and how they map to various available cloud offerings, both AWS-like or “Enterprise”?

            I am trying to understand the reasons why you think that there’s no need for Service Providers to offer “Enterprise-grade” IaaS.

            I do understand why “Enterprise-grade” model *will* be dead after a few years, but I don’t understand why would one ignore it today.

            Have a great weekend, too.

            P.S. I quite agree with the views on enterprise and private cloud’s “places in life” described here: http://blog.gardeviance.org/2011/02/private-vs-enterprise-clouds.html

            Cheers,

            – Dmitri

          • http://cloudscaling.com/blog randybias

            Both enterprise clouds and open clouds are “enterprise grade”.  Open cloud infrastructure can be as enterprise-ready as any other solution.

            It’s a question of architecture, not quality.  Enterprise cloud solutions are complex, difficult to scale, and solve problems for a completely different set of applications.  They have a place, but folks need to be realistic about what goes on them and what doesn’t.  What I’m seeing is that public cloud providers who offering enterprise private cloud solutions do ‘ok’, but not great in terms of adoption and business model.  They also primarily attract a different sort of buyer and application.  Enterprise apps and IT buyers, not cloud-ready apps and app-dev buyers.

            I’m not advocating ignoring enterprise clouds.  Cloudscaling is largely focused elsewhere, but others will work on enterprise private cloud solutions and I think that’s fine.

            Yes, Simon is right on.  Our business model reflects our commitment to a similar vision while also pragmatically understanding the interim transitions that need to occur.

          • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

            Great, the confusion has been resolved now :)

            One more question, if you don’t mind: do you have a feel for whether current size of the market for cloud-ready apps and app-dev is geographic region-dependent? I.e., would you say that it is significantly more advanced in US then some other regions?

          • http://cloudscaling.com/blog randybias

            Yes, I would say that cloud-ready is different in different geos.

  • Hameed

    We can debate as long as we want but if you are a big name and if you say some thing loud enough and repeat that a 1000 times, people just believe it.  Remember Sun Micro? They used to say their systems were ‘Open and based on Standards’ vs. Closed systems like Microsoft.  The party lasted as long as the customers found that to be valuable.

  • Pingback: Applied Software Architecture (paperback) | Software Architecture In Practice