The Top Three Things To Bookmark for the Juno Summit

Posted on by Azmir Mohamed

I can’t believe that it’s less than a month before the the upcoming Juno Summit.  As you start putting together your plans for the Summit, I wanted to highlight some items to look forward to from the Cloudscaling team.

#1: Four Cloudscaling sessions have been selected for the Summit.

We appreciate the explicit endorsement from the community – the topics reflect on our experience and leadership in the space – and we are very excited to share! Here is a recap of the four Cloudscaling sessions – you can simply add them to your Summit schedule by clicking on the links:

  • Hybrid Cloud Landmines: Architecting Apps to Avoid Problems (Randy Bias & Drew Smith): Application portability? Cross-cloud replication and bursting? These are hard issues with serious implications that we’ll examine in detail, digging into the subtleties between simply implementing a cross-cloud API and actually using it to get real work done. We’ll provide a set of recommendations on how to architect your application from the ground up to be hybrid cloud friendly.

  • Open Cloud System: OpenStack Architected Like AWS (Randy Bias): At Cloudscaling, we spend a great deal of effort operationalizing OpenStack for customers and ensuring it’s compatible with leading public clouds. In this session, we’ll detail the configuration and design decisions we make in real world OpenStack deployments.

  • OpenStack Block Storage: Performance vs. Reliability (Randy Bias & Joseph Glanville): In this presentation, we’ll talk about selecting and designing the right kind of storage for your use case. We’ll also walk you through how we built our block storage solution in Open Cloud System, its design principles, and explain why it is a better version of AWS Elastic Block Storage (EBS) in several key ways. (I regret we have to pull the plug on this presentation due to time constraints!  Hopefully in fall.  Best, –Randy)

#2: For the second Summit in a row, we are sharing our booth space with Quanta.

The reason is simple – a significant majority of Cloudscaling customers use Quanta servers and storage equipment for their OpenStack powered clouds (and did I mentioned what such a great team they are to work with?). While OpenStack’s role is to ultimately abstract the underlying physical infrastructure, we have found Quanta hardware to be a perfect complement to elastic OpenStack-powered clouds. The Quanta team will be bringing a few racks of their datacenter products that are most optimized for building modern OpenStack clouds. Stop by our booth!

#3:Open Cloud System (OCS) product announcement.

OK, I know it’s a teaser. But it should be no surprise that Icehouse will be central to the OCS announcement but we have a few additional items up our sleeves to share including

  • New OCS management capabilities

  • Additional AWS compatible features

So, it’s an understatement to say that we will be busy between now and the Summit. But any opportunity to meaningfully interact with the community, our customers and our partners is worth its weight in gold. We look forward to seeing you in Atlanta!

Posted in OpenStack | Leave a comment

A Concierge in the Cloud

Posted on by Azmir Mohamed

Today at Interop in Las Vegas, Cloudscaling introduced our new Cloud Concierge Services. The idea for Cloud Concierge came about as we watched enterprises struggle to understand where to start implementing an OpenStack cloud solution. Between public clouds, private clouds and legacy technology, there were many interrelated and complex decisions companies have to make.

Cloud Concierge aims to reduce complexities and accelerate adoption of enterprise OpenStack cloud implementations. We achieve this by:

  • Building all-inclusive offerings catered to fixed budgets and timeframes,

  • Grouping the Cloud Concierge offerings in logical phases of OpenStack adoption,

  • Addressing the needs of key stakeholders in the enterprise, such as the infrastructure and application teams, who are transforming  IT.

More importantly, our Cloud Concierge Solution Architects will train IT professionals on new skills and guide them to insights needed to operate in a modern cloud era. In addition to getting the IT team started in the short-term, they’ll gain new learnings that will allow them to better serve their organizations for the long-term.

Adopting a cloud strategy is a must in today’s business world. An OpenStack decision will give companies greater agility, transform IT service delivery and create differentiated business value. Cloud Concierge will help you go from “Zero to Cloud” on time and on budget.

Get in touch, and let us know how we can help you get started. Learn more about our Cloud Concierge Services at www.cloudscaling.com/services/cloud-concierge-services/.

 

Posted in OpenStack | Leave a comment

Cloud Just Got Real – The Race Begins

Posted on by Randy Bias

This week, Google hosted its Cloud Platform Live event. Some people were a little surprised at my enthusiastic live twitter coverage for a number of Google’s announcements. I have been waiting for them to “go big or go home” for a while now. My biggest surprise was how long it took Google to get up to speed. My fundamental belief is that they waited so long because they realized what they needed to do to succeed had little to do with initial speed, but was more about long term momentum.

Let me explain.

There are Three Ways to be #1

I wish I could claim this is my original thought, but I heard it from someone else[1]. If you know a more canonical source, I would very much like to have it. Please send it along via twitter or comments.

The saying goes like this:

There are three ways to be #1 in a market:

  1. Being first
  2. Being best
  3. Being cheapest

Ideally, you would be two or more of these things, leading to market dominance.

If you were Google, entering into this market years after AWS has launched and gained the market traction they have, you might reflect on what your best strategy is. Maybe you know the above truism or maybe you just have good instincts. If you are playing to win and you can’t be first that means you need to be best or cheapest, but preferably both.

And that, in essence, I think boils down why Google took a while to get “in market” and really tried to nail the technology before getting very serious as we saw this week.

Public Cloud is A Development Engine Game

I have always thought that public cloud was less about technical features (e.g. VMs on demand, object storage, etc.) and more about building a world class development engine. As I noted on previous occasions, businesses like Amazon actually increase in feature velocity as they get bigger, not decrease. This is a relatively new phenomena only seen in web-scale businesses (and Apple).

Building velocity like this is less about technology and more about culture and organizational structure. I’m sure you have read the seminal The Mythical Man Month, which essentially says that as a development team gets larger your overhead on communication increases to the point where you actually move slower. The answer to this is typically moving to an agile model, but that’s only a partial fix. It is far more effective to make your development organization look like a set of loosely coupled independent startups with clear targets for success and clear accountability. In that way teams run their own business.

This also maps to the actual underlying technology structure of the business and is the general “a-ha” moment that large scale web businesses collectively had. Here’s a 2007 ZDnet article talking about Amazon.com’s (the retail site) services oriented architecture.

Google Cloud Platform’s Announcements

I don’t really need to recap the various announcements this week. You can find that in various places. What I do want to highlight is the two underlying trends that these announcements represent:

Google thinks this is a game for the hearts and minds of developers of new apps
Google is doubling down on it’s own development engineering might first

This is reflected in the Wired article that came out yesterday entitled Google’s Bold Plan to Overthrow Amazon as King of Cloud.

The key quote is here:

“We will spend the majority of our development efforts on this New World,” wrote [Urs] Hölzle. “Every developer will want to live in this world…and it’s our job to build it.”

More succinctly what Urs is saying is that Google’s strategy is to build a developer advantage inside Google in order to enable developers outside of Google to have cloud services (platform and infrastructure) that they will love.

Hints of this can be found in the more nuanced technical announcements yesterday such as the support of integrated build/test pipelines through a combination of Google Compute Engine (GCE) + Google App Engine (GAE) using the GAE Managed VMs offering.

Being Best AND Cheapest

Google’s approach to their Cloud Platform has been the slow and steady buildup of a hard-hitting, quick-firing development engine that is capable of increasing the velocity of feature releases over time. That development engine has now reached escape velocity. There are only two other public clouds like this: Amazon Web Services and Microsoft Azure [2].

Google’s development engine shows they are targeting being Best AND Cheapest. There are a number of good reasons why they can accomplish those goals and perhaps I will deep dive into them in a future blog posting.

Consider this … If it’s a three way horse race in public cloud with OpenStack for the private cloud then we need to accept that we are now living in a multi-cloud world. For me, this is a sure indicator of a rapidly maturing marketplace that delivers maximum choice to developers and enterprises.

To paraphrase: Cloud just got real.


[1] Marc Andreesen I think, but I can’t find the blog posting where I thought he laid it out. BTW, if you haven’t read Marc’s original blog postings on entrepreneurism and startups, you are really missing out. The archive is here.
[2] HP might develop this capability over time, but they need to seriously commit to the path, so there are significant question marks about their possibilities for success until their business is more stable.

 

Posted in Cloud Computing | Leave a comment

Pets vs. Cattle: The Elastic Cloud Story

Posted on by Randy Bias

Last night I presented at the Chicago DevOps Meetup along with @mattokeefe and @mfdii.  The presentation went very well and was warmly received.  It’s a bit of a revisit of some topics I have covered before, such as Pets vs. Cattle, the new kind of elastic cloud, and the architectural patterns involved with building these new kinds of clouds.  I did a major refresh though and am weaving the threads a bit differently than before.

I hope you enjoy it.

Pets vs. Cattle: The Elastic Cloud Story from Cloudscaling, Inc.
Posted in Cloud Computing | Leave a comment

The Case for Tiered Storage in Private Clouds

Posted on by Randy Bias

Fall of last year I wrote a controversial whitepaper detailing my concerns about how distributed storage was being marketed.  The blog introduction and the whitepaper were both entitled Converged Storage, Wishful Thinking, and Reality.  There was a certain amount of expected blowback from folks at RedHat and Ceph as well as more thoughtful replies from Dreamhost and Solidfire.  Some of the feedback was pure screed, some was worthy of considered thought, and much of it missed the mark.  I realized however that it was *my* failure to communicate clearly that caused a considerable amount of the feedback to be erroneous.

So I wrote an update to that whitepaper that I think helps clarify my thinking in this area.  Fortuitously, the recent Google outage really helps to emphasize what I am talking about.  If you haven’t been paying attention, you can find out more about it here and from Google themselves.  In a nutshell, Google, who runs a 3-4 nine operation for most services (meaning 99.9-99.99% uptime), had a 35-minute outage due to a software bug.  It’s noteworthy that an event like this is so heavily scrutinized given that the vast majority of businesses struggle to achieve three nines of uptime.

More importantly, this outage highlights what I was trying to say in my original paper, which is that large homogeneous software systems are inherently dangerous to uptime.  Systems fail because of hardware failure, operator error, or software bugs.  Traditional high availability (HA) pairs solve for the hardware failure problem, but typically ignore operator error or software bugs.  Newer distributed software systems *still* predominantly solve for hardware failure, ignoring the more likely failure scenarios of operator error and software bugs.  This is a major problem.

There are a number of ways to solve these problems, including but not limited to: running more than one kind of software system (i.e. moving to a more heterogeneous set of systems), using sharding techniques, and applying disaster recovery techniques.  All of these approaches are essentially doing the same thing, which is limiting the failure domain size.  Not having fault isolation means you will see cascading failures as AWS has seen several times when it violated some of these basic principles. Operator errors combined with software bugs caused massive EBS failures, which spilled across Availability Zones because the EBS control plane spanned the AZes.

This is my problem with how distributed storage systems are being marketed.  I love distributed storage.  I think it has a place in the datacenter.  Cloudscaling is currently evaluating our options for integrating open source distributed storage software in our product.  The problem is that it’s place is not to run everywhere in the datacenter and the marketeers at various storage startups who believe so and market so-called “unified” storage solutions are really setting everyone up for failure.  There is no spoon.

For more details, here is my updated storage whitepaper, now entitled The Case for Tiered Storage in Private Clouds, that hopefully furthers the conversation on the appropriate best practices and patterns to use for deploying storage in private clouds today.

In the interest of transparency, I will say that this new revision was kindly reviewed by the team at SolidFire, who have a similar viewpoint to mine.  Much thanks to Dave Wright, Dave Cahill, and John Griffith (PTL for OpenStack Cinder) for their feedback.

Posted in Uncategorized | Leave a comment

Vote Now: Awesome Proposals for the Spring OpenStack Summit

Posted on by Randy Bias

Voting is now open for the Spring 2014 OpenStack Summit in Atlanta!

Cloudscaling (and our friends at Mirantis) have submitted a variety of great session abstracts, from Randy Bias’ walkthrough of hybrid cloud landmines to avoid when architecting applications to Boris Renski’s discussion on updating the OpenStack mission statement to disrupt large player competitive barriers and keep the stack open for innovation.

Below, we’ve summarized each talk and provided a link to its page on the voting site. As always, you must be a member of the OpenStack Foundation to vote. If you’re not already a member, just click here to join as an individual member for free. You get to shape the OpenStack Summit agenda by voting up the sessions that you’d like to see.

We look forward to seeing you in Atlanta!

Cloudscaling Session Proposals

  • (Vote Here) Open Cloud System 3.0: OpenStack Architected Like AWS (Azmir Mohamed) - At Cloudscaling, we spend a great deal of effort operationalizing OpenStack for customers and ensuring it’s compatible with leading public clouds. In this session, we’ll detail the configuration and design decisions we make in real world OpenStack deployments.

  • (Vote Here) OpenStack Scale-out High Availability: Scaling to 1,000+ servers without Neutron (JC Smith + Randy Bias) - The default OpenStack networking models don’t make sense in a modern datacenter. Similar to the approach taken by HP, CERN and other deployments, we’ll detail how we replace the default networking model through standard plugins in order to deploy a fully scale-out networking model that can support 100s of racks with incredibly high performance.

  • (Vote Here) For Cloud Operators: Hands-on With OpenStack for Elastic Clouds (Sean Winn) - In this session, we will provide a condensed version of our 2-day OpenStack Bootcamp workshop with an operator-centric, hands-on approach.

  • (Vote Here) ZFS on Linux + OpenStack: Flash Performance at Spinning Disk Prices (Joseph Glanville) - Join us for this walkthrough of ZFS on Linux (ZoL) where we’ll cover the state of ZoL along with the related OpenStack plugins and drivers. You’ll leave with a clear understanding of why multi-tenant cloud workloads are perfect for ZFS as well as how to measure and tune performance.

  • (Vote Here) Voice of the User: Running OpenStack in Production (Drew Smith + Ryan Lane) - Join us for an analysis and interactive discussion of real world OpenStack deployment issues. Along with guest speaker, Ryan Lane, who is running an OpenStack based cloud, we’re going to combine a review of data from our field and support requests with war story examples of “when it goes wrong”.

  • (Vote Here) OpenStack Block Storage: Performance vs. Reliability (Randy Bias + Joseph Glanville) - In this presentation, we’ll talk about selecting and designing the right kind of storage for your use case. We’ll also walk you through how we built our block storage solution in Open Cloud System, its design principles, and explain why it is a better version of AWS Elastic Block Storage (EBS) in several key ways.

  • (Vote Here) iSCSI Performance Testing & Tuning (Joseph Glanville) - In this session, we’ll explore the pros and cons of different iSCSI implementations within OpenStack Cinder deployments. We’ll then look at performance tuning options for iSCSI as well as the effects of related architecture components such as using jumbo frames in the network, flash storage, hybrid storage pools and iSCSI multi-pathing.

  • (Vote Here) Emulating A Large Scale, Multi-Rack Cloud on One Server (Mike Lang) - In this presentation we’ll introduce a new open source tool that allows us to emulate a multi-rack, multi-server, OpenStack deployment on a single server, allowing complete cloud architectures to be tested. We’ll also recommend how this tool can be used to expand the OpenStack continuous integration system and possibly serve as a mechanism for testing a variety of OpenStack reference architectures within complex, multi-rack deployments.

  • (Vote Here) Hybrid Cloud Landmines: Architecting Apps to Avoid Problems (Drew Smith + Randy Bias) - Application portability? Cross-cloud replication and bursting? These are hard issues with serious implications that we’ll examine in detail, digging into the subtleties between simply implementing a cross-cloud API and actually using it to get real work done. We’ll provide a set of recommendations on how to architect your application from the ground up to be hybrid cloud friendly.

  • (Vote Here) Tinker: A Tool for Network Configuration Management at Cloud Scale (Abhishek Chanda) - Managing network configuration in a cloud deployment can be a major pain. In this session, we will introduce tinker, a tool to generate and manage network device configuration using existing metadata to automatically and reliably generate configurations for all network devices in a cloud.

  • (Vote Here) OpenStack and the Google Compute Engine APIs (Alex Levine + Randy Bias) - It’s a hybrid cloud world and OpenStack powered public clouds won’t be everywhere, so it makes sense that we have a rich ecosystem of compatibility APIs in addition to the native OpenStack APIs. Come learn about the Google Compute Engine (GCE) APIs we recently built and submitted back to StackForge, including how they were built, how to add them to your own OpenStack deployment, and how to test them.

Mirantis Session Proposals

  • (Vote Here) Configuring Live Migrations for Workload Transitions (Damian Igbe) – This talk will review approaches to three fundamentally different live migration methodologies: block migration, shared storage migration, and migrating instances booted from Cinder volumes (i.e., volume booted migration). It will compare these three approaches and discuss factors to consider when deciding which methodology to use in different scenarios. We will also review practical (and sometimes critical) steps to ensure that both the cloud and its workloads are properly configured for live migration, along with important factors that can make or break smooth implementation and operation of live migration.

  • (Vote Here) Step One, Launch Openstack; Step Zero, Pick your hardware (Gregory Elkinbard) – The power of OpenStack software implementation, deployment and operations all cross the critical path of hardware selection. In this presentation, we’ll present a distillation of the key factors we have applied at Mirantis through dozens of highly successful OpenStack deployments on how to select an optimal hardware platform for your OpenStack cloud.

  • (Vote Here) Source Based Packaging: The Missing Ingredient of Continuous Delivery? (Dmitry Borodaenko) – This talk will explore the gap between source-based CI used by the OpenStack community and package-oriented deployment favored by OpenStack distributions, and examine the state of tools aiming to bridge that gap. Attend and you’ll learn how to tick the OpenStack CI/CD checkbox without leaving the comfort of the package-config-service pattern and the safety of security updates from your favorite distro.

  • (Vote Here) Is it time to update the OpenStack mission statement? (Boris Renski) – This talk will take a closer look at various OpenStack community initiatives and projects that disrupt competitive barriers established by large infrastructure players. We’ll also examine how the OpenStack community is making OpenStack distribution vendors in their present form irrelevant in many respects.

  • (Vote Here) Preparing and Publishing an Application in the Openstack Application Catalog (Alexander Tivelkov) – This workshop will walk you through the steps required to prepare an application for publication. The presenters will pick a real-life application and demonstrate how to publish it in the Catalog. Attendees will see how to define the configuration parameters and dependencies, build the deployment and maintenance workflows, construct a user-friendly UI to control these workflows – and, eventually, how to publish all of it in the Catalog. And all of this without writing a single line of Python code or modifying a single bit of the application itself.

  • (Vote Here) How to Avoid Picking Problems that Openstack Can’t Solve (Evgeniya Shumakher) – To be effective promoters and developers of OpenStack, we need to be able to identify these cases (not all edge-cases, either) and have the firmness of conviction to guide prospective users away from OpenStack, when risks are so high that their results will be marginal or unsatisfactory. (And we need to look at these gaps, where they exist, and think hard about filling them.) Attendees will leave this presentation with a fuller understanding of how to pilot their organizations away from what may become lackluster OpenStack engagements, and towards opportunities that take best advantage of OpenStack as it now exists.

  • (Vote Here) Extending the Fuel Project to Accelerate and Simplify Deployment of OpenStack Drivers and Components (David Easter) – In this session, we’ll fully familiarize attendees with the (relatively simple) process of becoming an OpenStack contributor. We’ll go over the process roadmap, account registration, security and authentication, connecting with community online resources and documentation, OpenStack and related code repositories, projects and their taxonomies, configuring Git to work within OpenStack’s code-review process and committing changes, review and approvals. And we’ll also cover details of how and where to best patch Fuel and describe the Fuel API timetable, imminent component releases and Fuel’s project roadmap. Attendees – particularly developers – should come away from this session ready to join the OpenStack project, find resources, pick goals and quickly be productive, especially where contributing to Fuel is concerned.

  • (Vote Here)  Rally – Benchmarking OpenStack at Scale Using Your Own Hardware (Boris Pavlovic) – Benchmarking a huge distributed system like OpenStack raises the complexity bar significantly. Add to that the fact that OpenStack may be a moving target – with Continuous Integration/Continuous Deployment (CI/CD) keeping things in flux – and benchmarking becomes, as Spock might say, “a non-trivial problem.” This session will explore Rally’s high level architecture, the current state of the Rally project, and the Rally Roadmap. We’ll also cover how to use Rally to improve the quality of OpenStack (Dev case), to ensure that OpenStack clouds pass SLA (QA/DevOps case) and to help build CI/CD. Attendees should leave this session with a solid understanding of Rally and most of the info they need to begin integrating Rally to existing OpenStacks and architecting it into new deployments.

  • (Vote Here)  How Community can make OpenStack Customer Centric (Alex Freedland) – As a major OpenStack vendor, Mirantis has developed a set of best practices for welcoming and working with end-user customers, extending to them the benefits of OpenStack community participation, and conversely, obtaining from them guidance and actionable market intelligence. By discussing these best-practices, we hope to 1) help the OpenStack community reach out more consistently, and with better results, to real end-user customers 2) develop beneficial (but also non-burdensome) roles in the community, enabling and shaping participation by a range of parties whose main motive is to use rather than build the product and 3) explore structural tweaks that will enable better exchange of information between ‘style: end-user’ participants and members of the community who participate in more fundamental ways.

  • (Vote Here) Bare Metal Peer-to-Peer Provisioning at Scale (Herman Narkaytis) – In this talk, we’ll review the current state of bare metal provisioning in TripleO / Ironic and Fuel and model the scalability limits of alternate approaches and architectures. We will discuss the state of things in Icehouse, including what use cases are covered, the algorithm, and limitations. We will also explore whether plans for Juno can reverse diminishing scalability and drive bare metal provisioning closer to linear performance. Bare metal provisioning (versus creating virtual machines) has its own challenges, and after attending this talk, you’ll not only have a better understanding of the differences, you’ll also have a feel for possible ways to mitigate those challenges.

  • (Vote Here) Breaking the East-West Barrier: How to Combine a Flat Network, Security, Manageability and Killer Performance in Openstack (Pedro Marques, Przemyslaw Grygiel) – In this talk, we’ll outline a practical, step-by-step method for configuring and provisioning a 200-node OpenStack cluster using Fuel. Using a fully-functional physical infrastructure, we broke through very real bottlenecks in Openstack defaults and in tooling (the proof of a real bottleneck is that when you resolve it, it exposes the next bottleneck). In addition, we will enumerate typical mistakes and pitfalls.

  • (Vote Here) Securing OpenStack with Intel Trusted Computing (Christian Huebner) – Intel Trusted Computing allows comparing the current state of a compute host with a previously stored good state. This presentation will contain a general overview of Trusted Computing, OpenAttestation server and client deployment on Red Hat Enterprise Linux, the inner workings of trusted_filter.py and its attestation procedure against an OpenAttestation server and configuration and testing of OpenStack controllers and compute nodes for trusted execution. The presentation is based on a recent deployment of OpenAttestation with both trusted and untrusted compute nodes using RedHat Enterprise Linux 6.5 and OpenAttestation 1.6.

Posted in OpenStack | Leave a comment

GCE API – Available Now on OpenStack StackForge

Posted on by Azmir Mohamed

For Cloudscaling, the journey to the hybrid cloud begins (but does not end) with API compatibility between the OpenStack powered private cloud and leading public clouds such as Amazon Web Services (AWS). Cloudscaling customers run OCS, our OpenStack powered private cloud solution, because it enables them to burst to and repatriate from AWS. Leveraging an OCS private cloud as part of their hybrid cloud strategy provides agility, choice and flexibility to the application and infrastructure teams.

The AWS EC2 API has been part of the OpenStack Compute (Nova) since the project’s inception – Cloudscaling have been a beneficiary of this initial community investment and we have built a robust business around it. A typical AWS customer has invested heavily on AWS specific tooling and training so the cost of change is high. Cloudscaling OCS makes the transition to an OpenStack private cloud palatable for these enterprises because the same tools (Rightscale, Dell Cloud Manager, AWS CLI, etc) and training are fully compatible. Customers do not have to start from scratch when transitioning from AWS to OpenStack using OCS.

However, we have always felt that the AWS API was not the end all be all for our customers and the broader OpenStack community. Today, we are changing that by making the GCE API available to the OpenStack community via StackForge. We believe the GCE API provides the same benefits as the AWS EC2 API to the community – agility, choice and flexibility .

More About StackForge

For those not familiar with StackForge, it’s the OpenStack incubator – it allows the broader community to see contributions and kick the tires on OpenStack related projects. StackForge is also the way that OpenStack related projects can consume and make use of the OpenStack project testing and continuous integration (CI) infrastructure. The ultimate goal of any StackForge project is the same – inclusion into OpenStack. We are passionate about GCE and why we have expended considerable energy to make the API set available on StackForge.

How passionate? For starters, take a look here. As you can see, Cloudscaling is keeping great company in StackForge – IBM, HP, RedHat and Mirantis are among the stable of companies that have contributed code to it. Our GCE API contribution is also significant – more than 27,000 lines of code broken across the GCE API, JSON with API definitions, OpenStack Commons, Unit and Tempest Tests. Finally, our GCE API contribution is covered under the Apache Project Licence (APL) v2, just like OpenStack.

Take GCE API For A Spin

Get started with GCE API here – simply enable it as a service on your Grizzly or Havana based OpenStack cloud. It’s also enabled on OCSgo, our evaluation private cloud where you can also experience all the functionality available in the latest OCS releaseWe are initially validating functionality using gcutil and will include other GCE compatible tools in the future. The Cloudscaling team believes we have done all the right things to make GCE API another choice in OpenStack. What do you think?

UPDATE: Broken link fixed to the repos.

Posted in OpenStack | Leave a comment

OCS 2.6 is Now Available !

Posted on by Azmir Mohamed

OpenStack is a powerful way of providing and consuming IaaS to achieve scale and agility, but its very nature as an open source project raises IT management challenges. Removing some of the implementation complexity and management obstacles from the equation is vital to enterprise adoption. Cloudscaling is committed to advancing OpenStack to meet the needs of the enterprise through advanced features in our core product Open Cloud System (OCS).

To that end, we are excited to announce that our latest release – Open Cloud System (OCS) 2.6 – is available today. This release marks a key milestone in the evolution of OpenStack, making it easier to deploy and consume.

The major features in this release include

  • Our control plane is radically simplified since all cloud services now run on the OCS Clustered Zone Nodes. Critical cloud services – including zone management, OpenStack and EC2 API endpoints, resource schedulers, logging, monitoring and secure operator access – are distributed and load balanced across the Clustered Zone Nodes. Starting with a minimum of three Zone Nodes in HA mode, the cloud can sustain multiple Zone Node failures and still continue operating. The same architecture can be scaled down for small deployments since a single Zone Node can run in a non-HA configuration. Adding capacity and redundancy is as easy as adding a pair of Zone Nodes to the pool.
  • OCS Block Storage performance is improved across all major performance metrics. How did we do it? First, we split the read (L2ARC) and write (ZIL) caches onto optimized SSDs. Then we transitioned to using Linux IO (LIO), which is natively supported in OpenStack. The results?
    • Block Storage performs consistently across sequential & random read and write operations. This allows apps with sensitive read and write requirements to be easily deployed in the private cloud.
    • Block Storage performs at an order of magnitude faster than AWS EBS – we measured in excess of 40,000 IOPS for 4K blocks. For enterprises that cannot get the performance they crave from AWS now have another reason to repatriate their apps to the private cloud.
  • Support for Dell servers using the Intel “Ivy Bridge” processors.
  • Dell Compellent SC280 JBOD. This high density JBOD provides a great alternative for customers who want to build a high density object storage solution.
  • Dell Force10 networking gear is now an option so Dell customers and reseller partners can spec out an “all Dell” cloud infrastructure stack for OCS pilot and production deployments.
  • OCS System Profiler is a feature that discovers and validates hardware for OCS deployments. Our system can now “see” the hardware it’s installed on and hands off this information to the various subsystems to use. System Profiler simplifies inventory of the underlying hardware so it accelerates all types of OCS deployments.
  • And finally, we are unveiling OCS Web UI – our single pane of management for the Enterprise Cloud Administrator. Dashboard is a service that also runs on the Clustered Zone Nodes and uses the OCS Cloud Operator API – so it’s fully integrated into our system.

           Initially we are addressing two key use cases with our Web UI:

  • Initial setup
  • Moves/add/changes of users and resources

But it does not end there. We are investing heavily on this graphical user interface        (GUI) because it is a critical component of how enterprises can realize the promise of life cycle managing hundreds of cloud nodes per cloud administrator. You can expect the list of capabilities and use cases to expand in the coming months. Visit our Resources page to learn more about our latest release.

In addition to OCS 2.6, Cloudscaling’s value extends through our Professional Services and Support teams. Visit our Cloud Concierge and Training pages to learn more about how we can help you put OpenStack to work for your business.

Posted in OpenStack | Leave a comment

Google is Amazon’s Only Real Competition

Posted on by Randy Bias

Recently, Barb Darrow wrote a note to Google Compute Platform, suggesting “8 Things it Could do to Freak Amazon out.” As long term readers know, I called out Google’s entry into the public Infrastructure-as-a-Service (IaaS) a while ago. My extensive experience at GoGrid and in the service provider and hosting business, provides me with a deep understanding of the public cloud market’s underlying business models and technologies. Cloudscaling also created the Google Compute Engine (GCE) APIs for OpenStack, which were finally submitted and made available via StackForge just today (more on that later).

Barb’s recommendations to Google were a good start, but I would like to expand a bit on her original article. Here are my thoughts.

8 Things GCE Could do to Freak out Amazon

#0 – sub-hour payment increments

This is not in the official list as it seems to be taken for granted that GCE already does this (they do) and that AWS should be concerned (they shouldn’t). While sub-hour payments sound good as it provides greater granularity in billing, it’s doubtful that most people will shave off any significant dollars. The main reason is that an hour of granularity is reasonably good and that most workloads expand during the day and contract at night following the general pattern of daily Internet traffic. The secondary reason is that when you combine server failure detection times with new VM spin up times and initial provisioning times, you are going to be well into a full hour. We simply aren’t at a point where the majority of apps spin up VM capacity rapidly (<1 minute) and spin it back down quickly. Elasticity is measured in hours, not minutes right now. That will change over time, but sub-hour payment increments are a premature optimization and marketing differentiation more than a reality for most customers right now.

#1 – Provide Reserved Instances

Makes sense. Couldn’t agree more. I’m sure they are hard at work thinking about it, but this is not a very big challenge so I’m sure Google will get to it when it makes sense to them and as Barb says it’s likely GCE is doing this on a case by case basis already.

#2 – More Managed Service-y Services

Google was churning out developer friendly services before Amazon even launched EC2 and it’s been cranking them out just as consistently. If you actually look at the Google dashboard you will be stunned at their ecosystem of services. It already eclipses AWS. I think it’s a mistake to look at Google as having to “catch up” to Amazon. What really happened is that Google started at the application layer and has been working down towards infrastructure, while Amazon started at the infrastructure layer and has been working up towards applications. Neither is really “ahead” of the other in the broader sense, although if you pick specific services, clearly their respective “leads” are relative to where a given services is in their “stack”. If it’s closer to the application layer, Google’s ahead, closer to infrastructure, Amazon is ahead.

#3 & #4 – Parlay Search / Get Louder

Not much to say here. These seem self-evident and I’m sure we’ll see more from Google given how they have been accelerating recently.

#5 – Offer More and Different Types of Instances

Makes sense. Customers want choice and variety. I am certain we will see choice increase over time. I will respectfully disagree with the smart and talented Sebastian Stadil however. It’s actually a mistake to provide live instance resizing. This is one of those kinds of suggestions that makes a *ton* of sense from the cloud end-user perspective, but actually has a negative impact for the public cloud provider. Live instance resizing isn’t strictly technically hard. Guest OSes support “hot plug” of RAM, CPUs, and other virtual hardware components and most hypervisors also support this. The problem however is that if you allow arbitrary instance resizing it’s very difficult or impossible to provide QoS and resource SLAs on a per instance basis. You also stand a very good chance of blowing up your business model by not having an even “bin packing” of instances on hardware. Most of these clouds need to run at around 80% sold capacity for the business model to work out. Instance resizing could cause that to go as low as 40% if you try and maintain QoS/SLAs, which would destroy the profitability of the public cloud itself. This is also more of a “pets” approach than a “cattle” approach.

For example, imagine you have a piece of hardware 4 cores (4c) and 4GB RAM (4g) that hosts 4 VMs, each of which get 1c/1g a piece (I’m simplifying here so bear with me). Now, each VM is roughly guaranteed one core each. This is an oversubscription ratio of 1:1 (AKA “no oversubscription”). If each customer suddenly resized these VMs to 4c/1g, that oversubscription ratio would change to 4:1, meaning that if all were banging away with their CPUs they would still only be getting roughly 25% of what they asked for, or one core. You can’t provide strong performance guarantees if you allow instance resizing. Now, some of this could be addressed by dynamically reallocating workloads via live migration on the backend and building a sophisticated scheduler to allow “re-packing”, but it would get very challenging in the situations where there simply wasn’t enough immediate capacity to service a request. The reality of the situation is that this is more trouble than it’s worth and customers would be better off learning how to dynamically spin up a new VM and scale horizontally. The new model is cattle, not pets.

I wrote up a detailed example of how oversubscription impacts cloud performance in late 2009.

#6 – Add More Datacenters

No comment. Google has a global reach and will use it. Nuff said.

#7 – Offer Virtual Machine Image Import/Export

Here here. Every cloud needs more of this. I’ve long said that even OpenStack Glance should be working more in this direction.

#8 – Do All of the Above But Faster

Google’s development engine is as fast as Amazon’s and in fact, you could argue that Google pioneered the agile webscale development model. This really comes down to a question of semantics. If you say that public cloud starts with infrastructure and then expands, then Google is chasing Amazon. If you say that public cloud starts with application services like Google Mail/Search/Apps, then *Amazon* is chasing Google, moving towards application level services. The reality is that either entry point is still “cloud” and that Google and Amazon have been inexorably moving towards each other. Google is the only major competitor to Amazon in this regard already. The only possible exception I would make being Microsoft, but they have their own internal problems to resolve before they are a serious contender.

Look at the number of developer centric APIs Google already has (this is a partial screenshot as the complete list is too long to fit):

google-dev-apis-screenshot

There are over *75* developer APIs across all of Google’s cloud services. Perhaps Amazon should start moving faster? :)

What Was Missed: Google’s Networking Advantage

Interestingly, one of Google’s biggest technical advantages was overlooked. For the past decade Google, Yahoo!, and Microsoft have been buying up dark fiber and deploying it between their datacenters. These folks, particularly Google have massive nation-wide networks capable of Terabit speeds (1000s of Gigabits). These networks interconnect Google datacenters. By some measurements Google is the 2nd or 3rd largest ISP in the USA.

Dark fiber doesn’t grow on trees. It’s incredibly costly to deploy and as what was in the ground has been sold off, it has become progressively more expensive. If you own the dark fiber and light it yourself, you can continue to push more bandwidth across the same strands by using new DWDM gear on either side. If you *don’t* have dark fiber, you only have access to “lit fiber” that limits how much traffic you can push across.

Google and Microsoft have massive structural advantages in their dark fiber networks and can not only have massive bandwidth between datacenters, but also have it at a very low cost. And Amazon cannot “come from behind” to make up the difference as supplies of dark fiber are now severely limited until the next wave of fiber buildouts and no one has any idea when that will happen.

What can Google do with this? Imagine the ability to have all Google Cloud Storage geo-replicated for free bandwidth costs. With so much bandwidth, moving huge VM images (100+ GB) across the network no longer sounds crazy. What about having your “regions” be as big as the entire eastern or western seaboards? All of this is possible if you have a big, fast, cheap WAN, which Google has and Amazon will never be able to achieve.

It’s Going to be a Two Way Horse Race

Exciting times in cloud land. Amazon Web Services finally has a credible competitor in Google’s Cloud Platform. I’m glad that others are beginning to see the light and I am still hopeful that Microsoft comes from behind and shows up to the party. In the meantime, watching Amazon and Google compete should provide plenty of entertainment and collectively we can all help the cloud revolution happen faster.

Posted in Cloud Computing | Leave a comment

Cloudscaling Webinar Series: 4 new installments

Posted on by Teri Elniski

In the modern world of IT the need for scalability and agility is undeniable. Cloud computing with OpenStack is a powerful way of providing and consuming IaaS to achieve scale and agility, but its very nature as an open source project raises IT management challenges. Removing some of the implementation complexity and management obstacles from the OpenStack equation is vital to enterprise adoption.

For enterprises to confidently migrate – as a matter of course – to OpenStack-powered IaaS, six requirements must be met. 1) High Availability, 2) Robust Management, 3) Open Architecture, 4) Scalable and Extensible, 5) Hybrid Cloud Federation, 6) Global Support and Services

The Cloudscaling webinar series will shine the light on the essential technical and business components required for organizations to confidently adopt enterprise-grade OpenStack powered clouds. Each webinar will include a live Q&A with the individuals at the center of the OpenStack movement. This four-part webinar series is now open for registration.

Bringing OpenStack into the Enterprise

Date: Thursday, January 23, 2014

Register Now


OpenStack 101: How You Can Take Advantage of Lost Cost, High Capacity Storage

Date: Thursday, February 6, 2014

Register Now


Modernizing Networking for the Cloud Era: VPC and the Hybrid Cloud

Date: Thursday, February 27, 2014

Register Now


Cloud Compatibility: Developing Private with Hybrid in Mind

Date: Wednesday, March 12, 2014

Register Now

Posted in Cloud Computing | Leave a comment

 

← Older posts