This really comes down to a particular TLA in use to describe grid: High Performance Computing or HPC.  HPC and grid are commonly used interchangeably.  Cloud is not HPC, although now it can certainly support some HPC workloads, née Amazon’s EC2 HPC offering.  No, cloud is something a little bit different:  High Scalability Computing or simply HSC here.

Let me explain in some depth …

Scalability vs. Performance
First it’s critical for readers to understand the fundamental difference between scalability and performance.  While the two are frequently conflated, they are quite different.  Performance is the capability of particular component to provide a certain amount of capacity, throughput, or ‘yield’.  Scalability, in contrast, is about the ability of a system to expand to meet demand.  This is quite frequently measured by looking at the aggregate performance of the individual components of a particular system and how they function over time.

Put more simply, performance measures the capability of a single part of a large system while scalability measures the ability of a large system to grow to meet growing demand.
Scalable systems may have individual parts that are relatively low performing.  I have heard that the Amazon.com retail website’s web servers went from 300 transactions per second (TPS) to a mere 3 TPS each after moving to a more scalable architecture.  The upside is that while every web server might have lower individual performance, the overall system became significantly more scalable and new web servers could be added ad infinitum.

High performing systems on the other hand focus on eking out every ounce of resource from a particular component, rather than focusing on the big picture.  One might have high performance systems in a very scalable system or not.

For most purposes, scalability and performance are orthogonal, but many either equate them or believe that one breeds the other.

Grid & High Performance Computing
The origins of HPC/Grid exist within the academic community where needs arose to crunch large data sets very early on.  Think satellite data, genomics, nuclear physics, etc.  Grid, effectively, has been around since the beginning of the enterprise computing era, when it became easier for academic research institutions to move away from large mainframe-style supercomputers (e.g. Cray, Sequent) towards a more scale-out model using lots of relatively inexpensive x86 hardware in large clusters.  The emphasis here on *relatively*.

Most x86 clusters today are built out for very high performance *and* scalability, but with a particular focus on performance of individual components (servers) and the interconnect network for reasons that I will explain below.  The price/performance of the overall system is not as important as aggregate throughput of the entire system.  Most academic institutions build out a grid to the full budget they have attempting to eke out every ounce of performance in each component.

This is not the way that cloud pioneers such as Amazon.com and Google built their infrastructures.

Cloud & High Scalability Computing
Cloud, or HSC, by contrast, focuses on hitting the price/performance sweet spot, using truly commodity components and buying *lots* more of them.  This means building very large and scalable systems.

I was surprised at the ISC Cloud Conference when I heard one participant bragging about their cluster with 320,000 ‘cores’.  Amazon EC2 (sans the new HPC offering) is at roughly 500,000 cores, quite possibly more.  And Google is probably in the order of 10 million+ cores.  Clouds built around High Scalability Computing are an order of magnitude larger than most grid clusters and designed to handle generic workloads, requiring hitting the price/performance sweet spot when building them.

Grid workloads can be very, very different.

Some Grid Workloads Drive the Grid Community
In talking to the grid community I learned that there are effectively two key types of problem that are solved on large scale computing clusters: MPI (Message Passing Interface) and ‘embarrassingly parallel’ problems.  I’m using terms I heard at the conference, but will use MPI and EPP (embarrassingly parallel problem) so that I can shorthand throughout the rest of this article.

MPI is essentially a programming paradigm that allows for taking extremely large sets of data and crunching the information in parallel WHILE sharing the data between compute nodes. Some times this is also referred to as ‘clustering’, although that term is frequently overloaded today.  Certain kinds of problems necessitate this sharing as the computed results on one node may effect the computed results on another node in the grid.  MPI-based grids, the de facto standard for most academic institutions, are built to maximum throughput and performance per system, including the lowest latency possible.  Most of them use Infiniband technology for example to effectively turn the entire grid into a single ‘supercomputer‘.  In fact, most of these MPI-based grids are ranked into the Supercomputer Top500.

An MPI grid/cluster, in many ways, looks more like an old school mainframe and technology such as Infiniband essentially turns the network into a high-speed bus, just like a PCI bus inside a typical x86 server.

EPP workloads, by contrast, have no data sharing requirements.  A very large dataset is chopped into pieces, distributed to a large pool of workers, and then the data is brought back and reassembled.  Does this sound familiar?  It should, it’s very similar to Google’s MapReduce functionality and the open source tool, Hadoop.  EPP workloads are very commonly run on top of MPI clusters, although some academic institutions build out separate or smaller grids to run them instead.

The majority of grid workloads are of the EPP type.  The diagram below shows this.

I had one person confide in me that “MPI power users drive grid requirements for vendors and assume that if their problems are solved, then the problems of [EPP] users are solved.”
This is interesting since these two types of workloads have different needs.

HPC vs. HSC
The reality is that High Scalability Computing is ideal for the majority of EPP grid workloads.  In fact, large amounts of this kind of work, in the form of MapReduce jobs have been running on Amazon EC2 since its beginning and have driven much of its growth.

HPC is a different beast altogether as many of the MPI workloads require very low latency and servers with individually high performance.  It turns out however, that all MPI workloads are not the same.  The lower bottom of the top part of that pyramid is filled with MPI workloads that require a great network, but not an Infiniband network:

In keeping with Amazon Web Service’s tendency to build out using commodity (cloud) techniques, their new HPC offering does not use Infiniband, but instead opts for 10Gig Ethernet.  This makes the network great, but not awesome and allows them to create a cloud service tailored for many HPC jobs.  In fact, this recent benchmark posting by CycleComputing shows that AWS’ Cloud HPC system has impressive performance particularly for many MPI workloads.

HSC designed to accommodate HPC!

Which brings us back.

The Moral of the Story
So, what we have learned is that scalable computing is different from computing optimized for performance.  That cloud can accommodate grid *and* HPC workloads, but is not itself necessarily a grid in the traditional sense.  More importantly, an extremely overlooked segment of grid (EPP) has pressing needs that can be accommodated by run-of-the-mill clouds such as EC2.  In addition to supporting EPP workloads that run on the ‘regular’ cloud some clouds may also build out an area designed specifically for ‘HPC’ workloads.

In other words, grid is not cloud, but there are some relationships and there is obviously a huge opportunity for cloud providers to accommodate this market segment.  At least, Amazon is spending 10s of Millions of dollars to do so, so why not you?

It can be confusing to understand how to scale computing systems, but it’s not rocket science.  There are really only two main axes of scale: out and up.  Closely related to the axis of scale is the general type of architecture: centralized or decentralized.  In this article I’m going to briefly revisit scaling and then talk about centralized vs. decentralized architectures.

The Axes of Scale

While ‘scaling’ is a popular topic, the scalability of a system is largely misunderstood.  You can scale ‘up’ (vertical) or ‘out’ (horizontal). Usually when people talk about scaling in the context of cloud computing, they mean a scale-out solution. This is because scale-up is not possible without control of the hardware, which you don’t usually have in a cloud computing scenario. Still, scale-up is a valid tactic for many situations such as scaling databases and fileservers.

Scale-out means use more of a single unit of resource (below), while scale-up means using fewer units while increasing the size of resource of each unit (up and right).

I believe scaling a system from this perspective is relatively well understood, but not necessarily the main ways in which scaling is achieved.

Centralized and Decentralized Systems

At the heart of most architectural or design decisions is whether to build a centralized or decentralized system. Centralized systems can generally be purchased off the shelf, come in high-availability (HA) pairs, are technically simple to operate, provide vertical scaling capabilities, and are delivered at a premium price per unit. Well-designed centralized systems provide very high uptimes and availability. They can also be prone to catastrophic failures due to mis-configurations.  This happens because they usually have a synchronized configuration and a misconfiguration in one is propagated immediately to the other. Given that misconfiguration is the number one source of failures or security breaches, this can be a major concern. Examples of centralized systems include: redundant load balancer pairs, NAS / SAN systems delivered as HA units, and centralized network switches.

Decentralized systems by contrast are scaled horizontally, can be technically complex to operate, usually written in-house or built from open source, and priced such that individual units are relatively cheap. Distributed systems tend to be highly resilient in the face of the failure of a single unit because configurations are not shared. A unit’s failure has no impact on neighboring units. At most, the overall capacity is affected. Examples include: shared-nothing web/app tiers, pools of virtualized servers, top of rack switch deployments, and peer-to-peer (P2P) systems.

Most web systems do not require an either/or decision as they use a combination of centralized and decentralized components. A good example is a typical 2-tier web application stack like Ruby-on-Rails. The web/app tier will be scaled-out and the databases will be scaled-up.  Despite the hype around cloud computing, this is still the norm.  Even on Amazon’s EC2, your typical web app starts with the smallest instance size possible and then when scaling limits are hit, is upgraded to the next biggest instance size.

The decentralized approach has been out of favor historically due to technical complexity and low margins.  Also larger enterprises tend to lean in the direction of simpler to manage, centralized, scale-up solutions. Vendors prefer to sell fewer, larger, high margin solutions while enterprises like technically simple solutions.  This is beginning to change as we can see with folks like Cloudera providing commercial support for decentralized data processing systems like Hadoop.

Centralized approaches have the distinct disadvantage of scale-up limits. Meaning you can only build a single server that is so large. If your growth needs fit neatly inside Moore’s Law, you’re in luck, if not, you’ll have to scale-out instead of up. At some point only a scale-out approach can continue to grow capacity.  This is why companies like Google and Amazon have chosen their particular ‘web-scale’ approaches.

The Secret in the Sauce
Secret sauce comes in many flavors and sizes.  Some sites need software, some need hardware, and others need special architectures.  More and more frequently cloud computing systems are used for those types of secret sauce that require some kind of batch processing.  Examples of batch processing applications include photo resizing and transcoding.  Cloud vendors like RightScale address the problem by providing a primary product that manages N-tier web applications and another one (RightGrid) that manages batch processing. Many also try to roll-their-own using technologies like Hadoop.

This diagram illustrates how these particular solutions work:

Secret Sauces of the World
Taking a look at some real-world examples will bring it home.  I’ve got a few below that I think will help us understand better.

• Scribd: Windows clusters for processing and transforming documents
• SmugMug: Photo resizing and storage
• FaceBook: Highly scalable memcached clusters, Haystack super scalable image storage system
• Runa: Real-time consumer purchasing analytics

In each of these cases, the web service needs to differentiate.  I’m particularly fond of Scribd’s secret sauce. The Scribd folks are largely a bunch of Linux & Ruby-on-Rails geeks who realized early on that the majority of document processing tools were on the Microsoft Windows platform.  So, they built their own Windows document processing clusters for secret sauce.  That’s pragmatic and clever.

Conclusion
Every web application needs something special to make it compelling to it’s chosen customer base.  This defines The Secret Sauce Problem that confronts every web service that grows to any significant size.  Although secret sauce can come in many forms, it is very common to be a batch processing application of some kind.  The interesting parts of any web application or service are not the web app itself, but the secret sauce.

If you don’t have the sauce, you’ll need it. Get some now.

In fact, they help in two ways: one by allowing an application to process more data using horizontal scalability (aka ‘elasticity’) and two by reducing load on the primary relational database and hence allowing you to go longer before ‘sharding‘.

Chris Wensel is the man when it comes to understanding Hadoop and he recently gave a couple of talks introducing Hadoop.  Here is one of them: