On-Demand Supercomputing

Except for the folks at Cray, most people are unaware of the unique requirements that set apart supercomputing infrastructure from cloud computing infrastructure. In its simplest form the difference is between latency and capacity. For business intelligence applications such as optimization and logistics many servers are required to solve a single problem, and low latency communication between the servers is instrumental for performance. The intuition behind this is easy to understand: a modern microprocessor executes 4-5 instructions per 250ps, and thus packet latencies of 10GbE, (between 5-50usec), are roughly equivalent to 100k to 1M processor instructions. If a processor is dependent on the results computed by another processor, it will have to idle till the data is available. Cumulatively, across a couple hundred servers, this can lead to peak performance that is only 1-5% of peak.

Supercomputing applications are defined by these types of tightly connected concurrent processes, putting more emphasis on the performance of the interconnect, in particularly the latency. Running a traditional supercomputing application on an infrastructure designed for elastic applications, such as AWS or Azure, typically yield slow-downs by a factor 50 to 100. Measured in terms of cost, they would cost 50-100 times more to execute on a typical public cloud computing infrastructure.

Most supercomputing applications are associated with very valuable economic activities of the business. As mentioned earlier, production optimization and logistics applications save companies like Exxon Mobil and Fedex billions of dollars per year. Those applications are tightly integrated in the business operation and strategic decision making of these organizations and pay for themselves many times over. However, for the SMB market these supercomputing applications offer great opportunity for revenue growth and margin improvements as well. However, their economic value is attenuated by the revenue stream they optimize; 10% improvement for a $10B revenue stream yields a $1B net benefit, but for a $10M revenue stream the benefit is just a $1M, not enough to compensate for the risk and cost that deploying a supercomputer would require.

Enter On-Demand Supercomputing.

In 2011, we were asked to design, construct, and deploy an On-Demand supercomputing service for a Chinese cloud vendor. The idea was to build an interconnected set of supercomputer centers in China, and offer a multi-tenant on-demand service for high-value, high-touch applications, such as logistics, digital content creation, and engineering design and optimization. The pilot program consisted of a supercomputer center in Beijing and one in Shanghai. The basic building block that was designed was a quad rack, redundant QDR IB fat-tree architecture with blade chassis at the leaves. The architecture was inspired by the observation that for the SMB market, the granularity of deployment would fall in the range of 16 to 32 processors, which would be serviced by a single chassis, keeping all communication traffic local to the chassis. The topology is shown in the following figure:

Redundant QDR IB Network Topology for On-Demand Supercomputing

The chassis structure is easy to spot as the clusters of 20 servers at the leaves of the tree. The redundancy of the IB network is also clearly visible by the pairs of connections between all the layers in the tree. The quad configuration is a two rack symmetric setup, one pair holding one side of the redundant IB network/storage/computes. So half the quad can fall away, and the system would still have full connectivity between storage and computes. To lower the cost of the system, storage was designed around IB-based storage servers that plugged into the same infrastructure as the compute nodes. QDR throughput is balanced with PCIe gen2 and thus we were able to deliver ephemeral blades that get their personality from the storage servers and then dynamically connect via iSCSI services to whatever storage volumes they require. This is less expensive than designing a separate NAS storage subsystem, and it gives the infrastructure flexibility to build high-performance storage solutions. It was this system that set a new world record by being the first trillion triple semantic database system leveraging a Lustre file system consisting of 8 storage servers (trillion-triple-semantic-database-record).

The provisioning of on-demand supercomputing infrastructure is bare metal, mostly to avoid any of the I/O latency degradation that virtualization injects. Given the symmetry between storage and compute and the performance offered by QDR IB, a network boot mechanism can be used to put any personality on the blades without any impact on performance. The blades have local disk for scratch space, but run their OS and data volumes off the storage servers, thus avoiding the problem of DR of state on the blades.

The QDR IB infrastructure was based on Voltair switches and Mellanox HCAs. Intel helped us tune the infrastructure, using their cluster libraries for the processors we were using, and Mellanox was instrumental in getting the IB switches in shape. Over a three week period, we went from 60% efficiency to about 94% efficiency. The full quad has a peak performance of 19.2TFlops and after tuning the infrastructure we were able to consistently deliver 18TFlops of sustained performance.

The total cost of the core system was of the order of $3.6M. The On-Demand Supercomputing service offers a full dual socket server with 64GB of memory for about $5/hr, providing a cost-effective service for SMBs interested in leveraging high performance computing. For example, a digital content creation firm in Beijing leveraged about 100 servers as burst capacity for post-production. Their monthly cost to leverage a state of the art supercomputer was less than $20k per month. Similarly, a material science application was developed by a chemical manufacturer to study epitaxial growth. This allowed the manufacturer to optimize the process parameters for a thin-film process that would not have been cost-effective on a cloud infrastructure designed for elastic web applications.

The take-away of this project is echoing the findings in the missing middle reports for digital manufacturing (Digital Manufacturing Report). There is tremendous opportunity for SMBs to improve business operations by leveraging the same techniques as their enterprise brethren. But the cost of commercial software for HPC is not consistent with the value provided for SMBs. Furthermore, the IT and operational skills required both to setup and manage a supercomputing infrastructure is beyond the capabilities of most SMBs. On-demand HPC services, as we have demonstrated with the supers in Beijing and Shanghai, can overcome many of these issues. Most importantly, it enables a new level of innovation by domain experts, such as professors and independent consultants, who do have the skills necessary to leverage supercomputing techniques, but up to now have not had access to public supercomputing capability and services.

The Goldman Sachs SaaS scorecard

The past couple weeks I have been peering over reams and reams of investment analysis regarding 'cloud computing'. It has been surprising to me that cloud computing as a concept is being misused to label disjointed offerings as the same. For example, viewing Google App Engine and Amazon EC-2 as conceptually the same is a fantasticly bad characterization and ripe for bad decision making for IT managers, software vendors, and investors.

In my mind, the issue of what cloud computing is or is not, is misguided to some degree and only useful for the software professionals that are leveraging the different offerings. And these folks should be sophisticated enough to recognize the different underlying technologies. From an end user perspective, what you can do with the end result of cloud computing is much more interesting. And in that regard, the offerings are much more easily understood. For example, using Google PicasaWeb or Smugmug to store your family's pictures on sharable, replicated, backed-up storage is an end user value you can measure.

In all this research I did come across a nice and simple scorecard that Goldman Sachs uses to educate its clients about one form of cloud computing: SaaS. Goldman Sachs uses the following characteristics to judge the value proposition of a SaaS vendor or on-premise ISV.

Does the application provide value as a stand-alone workflow or does it require extensive integration with other applications?

Integration is the biggest hurdle a SaaS provider encounters. When applications, and the workflows they automate, become more standardized with accepted APIs, this will become less of a hurdle but right now stand-alone value is the litmus test for success.

Does the application represent an industry best-known-method or does it require extensive customization?

Due to the fact that SaaS business models can only create value if they aggregate multiple users on the same software and/or hardware instance, customization dilutes the profitability.

Is the application used by a distributed workforce and non-badge employees?

This is clearly the driving force behind SaaS and other forms of cloud computing. In my mind, this is routed in a pattern that has been driving IT innovation for the past decade. Consumers have become accustomed to mobility of information in their personal lives. Universal access to email or travel itineraries is so natural that it is aggravating when corporate data systems don't provide the same sophistication. It is hard to have confidence in your IT department if they push applications on you that look and work horribly compared to the applications you use in your personal life.

Does the data managed by the application cross a firewall?

This is the security aspect of SaaS applicability. If much of the data comes from outside the firewall as aggregated information to help the business unit then this is a simple decision. Many B2B workflows have this attribute. If the data being manipulated are the crown jewels of the company, attributes such as security, SLA, accountability, rules and regulations become big hurdles for adoption.

Does the application benefit from customer aggregation?

This is the benchmarking opportunity of SaaS. If you host thousands of similar businesses then you have the raw data to compare these companies among each other, and possibly against industry wide known metrics.

Does the application deployment show dramatically lower upfront costs than on-premise solution?

Low upfront costs can defuse deployment resistance. This is very important attribute for a SaaS offering that is targeting the Small and Medium sized Business (SMB) segments.

Does the application require training to make it productive?

If it does it is bad news: internet applications need to do task extremely well particularly if your access to the application is through some seriously constrained interface like a smart phone or MID.

Can the application be adopted in isolation?

This is similar to the stand-alone requirement, but more focused on the procurement question for the SaaS solution. If the solution can be adopted by a department instead of having to be screened for applicability across the whole enterprise, it clearly will be easier to get to revenue.

Is the application compute intensive/interactive?

SaaS applications don't do well with large compute requirements mainly due to the multi-tenancy that is the basis of the value generation. If one customer interacts with the application and makes a request that pegs the server on which the application runs, all other customers will suffer.

Federated Clouds

Google's CEO Eric Schmidt defines cloud computing as the convergence of information. The cloud aggregates raw data, organizes the data so that it becomes valuable as information, and the information is accessable from any device anywhere. The organization of the data to transform it into information is where the "computing" term is justified in Cloud Computing.

So in a nutshell, clouds organize data to create value in terms of targeted information that can be sold or auctioned, such as advertising keywords, or the best price on an airline ticket. However, information truly is unbounded, and thus there will be many specialized clouds to add very specific value to specific raw data sets. For example, the data maintained in Amazon's cloud and its computational processes to create a searchable book store are very different compared to Google's Web Service or NASDAQ's Data Store.

Given the fact that not even Google can encapsulate all knowledge, diversity of information will lead to a commercially motivated federated system of clouds. Each cloud has its own optimized data organization capability to generate valuable information for profit. Amazon and Google will have opportunities for innovation that nobody else has due to their scale, but peripheral innovation will occur through aggregation, or mashups, of data residing in different clouds, thus creating a federation of clouds.

As far as the starting premise of what makes a cloud a cloud: convergence of information, the federation of clouds continues this premise, and thus can be seen again as a cloud.

Riches for SaaS providers

Many consultancy business are built upon serving the need for comparitive benchmarking of business operations. Ranging from strategy and efficiency, to more detailed metrics such as employee retention and productivity. Most of these consultancies use old-fashioned surveys and questionaires to gather this data which of course is fraught with data quality issues. Well, no more! There is a better way and it is through SaaS providers.

To generate the necessary economies of scale, SaaS is by necessity multi-tenant. Secondly, the dynamics of business haven't changed so SaaS providers need to race to critical mass in terms of installed customer base to continue to be relevant and generate free cash flow to drive continued innovation and roll out new functionality. Only the largest SaaS providers in a vertical will survive.

This creates the next monetization option for SaaS providers: business intelligence and benchmarking. The SaaS provider has brought together a wealth of companies all using the same business process codified in the SaaS functionality. Mining this data for trends and operational business metrics is just a small step. Global competition forced big companies to develop these business intelligence processes and they had the operational scale necessary. Small and medium business operation had to be aggregated to be able to generate this opportunity and this will be many times over more valuable than the SaaS functionality that the SaaS provider started with.

Cloud Computing Lingo

To understand all the marketing information that bear the label "cloud computing" it is helpful to have a quick glossary of terms.

Grid Computing

Grid Computing is a collaboration model. Locally managed resources are virtualized and aggregated in a larger, more capable resource. Grid computing is concerned with coordinating problem solving in virtual organizations and typically is associated with large and complex "Grand Challenge" problems.

In the scientific community, multi-institutional collaboration is required to have any hope of solving fundamental questions as they arise in high-energy physics, fusion or climate research. It is in this community that the world wide web originated to fulfill the need for seamless document access among geographically dispersed team members, and it is also the birthplace of the grid. In 1990 the first HTML communication took place at CERN, and in 1994 the first grid was put together around the Supercomputing conference as a mechanism for all participants to share data and models. It was dubbed I-WAY at that time but it was the starting point of research to try to find solutions for security, resource management, job control, and data caching that are central to grid computing.

Examples are: Terra GRID, Euro GRID

Haas, or Hosting as a Service

HaaS is a business service model. There are a lot of activities in modern business that are not core operational differentiators. These essential but peripheral services are better outsourced to specialists who can leverage economies of scale. Payroll management, shipping, and web presence are three examples of services that tend to be outsourced for most modern businesss, particularly small and medium sized businesses (SMBs).

Examples are: Startlogic, Hostmonster, Rackspace

SaaS, or Software as a Service

SaaS is a software deployment model. Application functionality is provided to the user through a web interface and the SaaS provider manages hardware and software operation and maintenance.

Examples: Salesforce.com, Webex, Netsuite

SaaS is generally associated with business software and marketed as a service to lower the cost of internally managed software. SaaS allows customers to lower the initial cost of software licenses and computer hardware to run on.

• Web Site Hosting and Web Application Hosting Services are probably the most ubiquitous instances of the SaaS model
• Customer Resource Management, or CRM, has many different instances, for example Salesforce.com, Siebel, or Coghead
• Completely integrated Enterprise Resource Management systems are provided by SAP, Oracle, Netsuite, Epicor, or Infor

Storage-as-a-Service and Computing-as-a-Service are slightly different in nature compared to SaaS since the latter provides application functionality whereas in the former two, access to a resource is sold or rented.

Commercially, SaaS has carved out many different useful services. Unfortunately, this has lead to a fragmentation of the market with the associated interoperability and economic lock-in problems. When selecting a SaaS provider the overriding question should be if you can move your data to other providers or bring it in house. SaaS becomes less interesting at a larger scale or if you want to extract business intelligence from your data. Plan for success but manage for failure. If the SaaS provider does not have a productive mechanism to get all the data out of the service, think twice before signing a contract.

PaaS, or Platform as a Service

PaaS is a software life-cycle model. Applications are developed, tested, deployed, hosted, and maintained on the same integrated platform.

Examples: Bungee Lab Connect, Comrange AppProducer

Web Services

Web Services represent anything that serves data, information, or access through a web browser. This is such a nebulous group of functionality that this term is more confusing then it is helpful. For example, Amazon's EC-2 is billed as Amazon Web Services, but really AWS rents you an appliance on which you can install your own machine image. That appliance can now do anything, from web site serving, to application serving, to data mining, to web indexing, to running your OpenOffice spreadsheet model. Google web services aggregate anything from email to calendaring to picture storage and of course web indexing, but it is distinctly different from Amazon's web services.

Web 2.0

Web 2.0 refers to the proposed second generation of Internet-based services where data services such as social networks, blogs, and wiki's are connected and add value to each other. Collaboration is central to this model and users generate information and police themselves.