Why is there so little innovation in cloud hardware?

With the explosion of data and the need to make sense out of it all on a smart phone is creating an interesting opportunity. Mobile devices need high performance at low power, and Apple seems to be the only one that has figured out that having your own processor team and IP is actually a key advantage. And the telcos will need Petascale data centers to manage content, knowledge management, and operational intelligence and the performance per Watt of general purpose CPUs from IBM, Intel, and AMD are at least two order of magnitude away from what is possible. So why is there so little innovation in cloud hardware?

The rule of thumb for creating a new chip venture is $50-75M. Clearly the model where your project is just an aggregation of third party IP blocks is not a very interesting investment as it would create no defendable position in the market place. So from a differentiation point of view early stage chip companies need to have some unique IP. And this IP needs to be substantial. This creates the people and tool cost that makes chip design expensive.

Secondly, to differentiate on performance, power, or space you have to be at least closer to the leading edge. When Intel is at 32nm, don’t pick 90nm as a feasible technology. So mask costs are measured in the millions for products that try to compete in high-value silicon.

Thirdly, it takes at least two product cycles to move the value chain. Dell doesn’t move until it can sell 100k units a month, and ISVs don’t move until there millions of units of installed base. So the source of the $50M-$75M needed for fabless semi is that creating new IP is a $20-25M problem if presented to the market as a chip and it takes two cycles to move the supply chain, and it takes three cycles to move the software.

The market dynamics of IT has created this situation. It used to be the case that the enterprise market drove silicon innovation. However, the enterprise market is now dragging the silicon investment market down. Enterprise hardware and software is no longer the driving force: the innovation is now driven by the consumer market. And that game is played and controlled by the high volume OEMs. Secondly, their cost constraints and margins make delivering IP to these OEMs very unattractive: they hold all the cards and attenuate pricing so that continued engineering innovation is hard to sustain for a startup. Secondly, an OEM is not interested in creating unique IP by a third party: it would deleverage them. So you end up getting only the non-differentiable pieces of technology and a race to the bottom.

Personally however, I believe that there is a third wave of silicon innovation brewing. When I calculate the efficiency that Intel gets out of a square millimeter of silicon and compare that to what is possible I see a thousand fold difference. So, there are tremendous innovation possibilities from an efficiency point of view alone. Combining it with the trend to put intelligence into every widget and connecting them wirelessly provides the application space where efficient silicon that delivers high performance per Watt can really shine AND have a large potential market. Mixed-signal and new processor architectures will be the differentiators and the capital markets will at one point recognize the tremendous opportunities present to create a next generation technology that creates these intelligent platforms.

Until then, us folks that are pushing the envelope will continue to refine our technologies so we can be ready when the capital market catches up with the opportunities.

And now something completely different: brain simulation

Our good folks at the national labs have been developing cloud computing for two decades, so what they are doing now is possibly an indication of what we'll be doing with the cloud a decade from now. Researchers at IBM Almaden have been working on the largest brain simulation to date; 1.6 billion neurons with 9 trillion connections. The scale of this endeavor still dwarfs the capacity and capability of any commercial cloud offering; the simulation uses roughly 150 thousand processors and 150TBytes of memory.

Just to provide a sense of the OPEX of such an installation: Dawn, a IBM Blue Gene/P supercomputer at LLNL, hums and breathes inside an acre-size room on the second floor of the lab's Terascale Simulation Facility. Its 147,456 processors and 147,000 gigabytes of memory fill 10 rows of computer racks, woven together by miles of cable. Dawn devours a million watts of electricity through power cords as thick as a bouncer's wrists—racking up an annual power bill of $1 million. The roar of refrigeration fans fills the air: 6675 tons of air-conditioning hardware labor to dissipate Dawn's body heat, blowing 2.7 million cubic feet of chilled air through the room every minute.

Given the fact that a real brain only consumes about 25Watts, clearly there is a lot of room for technology innovation. Silicon innovation however has come to a stand still with venture capital completely abandoning this segment. There are no VC firms in the US or EU that have any funds that target this vertical. It is rumored that Google is designing its own silicon now since no commercial chip manufacturers are providing the innovation that Google needs.