Governmental IT: Analytics is not a dirty word

Over at Smart Data Collective, Bill Cooper wrote a wonderful article on deep analytics. In particular, I liked his assessment on the resistance expressed by customers that can't see the forest for the trees.

I’ve watched many government agencies balk at the idea of data mining and complex analytics. They are concerned about switching to a new data architecture and the potential risks involved in implementing a new solution or making a change in methodology.

Having been there, I do understand their concerns, but fear of change is what’s holding government agencies back from being able to fully leverage the data that already exists to effect change at the local, regional, state and national levels. Analytics are the key to lowering costs, increasing revenue and streamlining government programs.

In my own government experience and now, looking at it from the other side, I have come to believe that government clients need to think about data the way the world’s top corporations do. Like all federal agencies, these companies already had huge repositories of data that were never analyzed – never used to support decisions, plan strategies or take immediate actions. Once they began to treat that data as a corporate asset, they started to see real results. The best part is that leveraging these mountains of data does not require a "rip and replace" approach. Inserting a data warehousing/data mining or complex analytics capability into a SOA or cloud computing environment can be very low risk and even elegant in its implementation. The potential rewards are immense!

That’s what’s needed in the government sector. We need to view analytics not as a dirty word but as a secret weapon against fraud and other challenges impacting all areas of the government sector.

I am a big believer that the future of the cloud consists of federated systems for the simple reason that large data sets are captive to their storage devices. Federated systems makes service oriented architectures (SOA) a natural architecture pattern to collate information. The fact that Google Gears and Microsoft Azure exhibit SOA at different levels of abstraction is clear evidence of the power of SOA. Add coarse grain SOAs to these fine-grained patterns and you can support federation and scale internal IT systems even if the core runs in Gears or Azure.

Interactive Map of cloud services

Appirio, a company that helps enterprise customers leverage PaaS cloud platforms such as Salesforce.com and Google Apps, put a nice interactive navigator on their website.
The Appirio cloud computing ecosystem map aims to provide more clarity in the fast evolving cloud services market. It tries to help enterprise decision makers to accelerate their adoption of the cloud by trying to provide a standard taxonomy.



Ryan Nichols, head of cloud strategy at Appirio, states: "The cloud ecosystem is evolving so quickly that it's difficult for most enterprises to keep up. We created the ecosystem map to track this evolution ourselves, and have decided to publish it to help others assess the lay of the land. With broader community involvement, we can create a living, breathing map where anyone can access, drill down and interact with dynamic information. This will bring some much-needed clarity to the cloud market."

Unfortunately, since the map is geared towards the enterprise customer it ignores all of the innovation that is taking place in the mashup, programmable web, and mid-market products, such as Zementis ADAPA in the Cloud. Given the new ways in which the cloud enables new application architectures and services, the enterprise market is the worst indicator of the evolving cloud ecosystem.

PC sales decline

In his post PCs at a Crossroads Michael Friedenberg reports on IDC's measurement of the PC marketplace. From the article:

"Case in point is the PC market. Market researcher IDC reports that 2009 will be the first year since 2001 where PC shipments will decline. I believe this drop is driven by a more rapid intersection of the cyclical and the systemic as the PC value proposition is challenged and then transformed. As Intel CEO Paul Otellini recently said, "We're moving from personal computers to personal computing." That comment signals Intel's way of moving into new markets, but it also acknowledges that the enterprise PC market has arrived at a crossroads."

Cloud computing is one development that is dramatically changing the desktop and server eco-system. The performance of a single server or desktop hasn't kept pace with the computational needs of modern science, engineering, or business. Cloud computing moves away from capital equipment to the ability to procure just the computational output AND at infinite scale for most use cases. Most desktops are idling most of the time, but are too slow to get real work done when you need it. This is pushing the work towards elastic resources that are consumed as you go. If the browser is all you need, then a move towards server consolidation and thin clients is not far behind.

Amazon Virtual Private Cloud

Yesterday, Amazon introduced Amazon VPC. It enables logically isolated compute instances and a VPN tunnel to connect to internal data center resources. The architecture is straight forward and Amazon's blog post depicts is as follows.

But the implications of VPC are far reaching; there are no real hurdles left to leverage Amazon's cloud except for limited and costly Internet bandwidth. Amazon's offering is morphing into a very flexible IaaS with some content delivery network features that are great for geographically dispersed small businesses. I am thinking particularly companies such as Stillwater that are differentiating through high domain expertise. Global talent cannot be bound to a small locale such as Silicon Valley anymore. We are connecting researchers in US, EU, Middle East, and Asia and with offerings like this we can create a development process that follows the moon that rivals the mega-vendor infrastructures. We do not need to uproot any folks to make this happen. These are exciting times we live in that can really unleash the creative spirit of the world.

Cloud Computing Taxonomy

I found this wonderful graphic created by Peter Laird in his blog.



Peter's blog has all the descriptions of the buckets.

The Public Cloud bucket is heavily underreported. There are roughly about 1200 public data centers in the US alone that are quite happy to rent you a server or cabinet. There are a host of data center market places that will connect you to a data center provider. Here are a few:

Find a Data Center

Data Center Knowledge

Data Center Marketplace

In particular, the telecom companies, like 365 Main, SuperNAP, Qwest, Verizon, Level 3 Communications are quite happy to sell you connectivity AND servers and are perfect for large cloud deployments that need geographic spread and high bandwidth.

On-demand pricing for Windows Azure

InformationWeek's Paul McDougall reports on Windows Azure pricing and it provides confirmation that Microsoft is transitioning its boxed software business into a service business.

Paul's assessment:
"Azure is the latest sign that Microsoft is eyeing the Web as the primary delivery mechanism for software and services. On Monday, the company said it planned to make a version of Microsoft Office 2010 available to consumers over the Internet at no charge. It plans a similar offering for businesses."

In my mind there is still one piece missing for productive cloud computing and that is the seamless integration of the client. None of the big vendors are particularly keen on solving this problem since it diminishes their economic lock-in. But users create, use, and transform data and information on their clients and data needs to seamlessly flow between the client and the cloud. This flow in my mind is best managed by the OS, or a tightly integrated run-time. You see many of these service components show up in the mobile platforms, but the PC ecosystem is lagging here.

Netbooks and the cloud

Dana Blankenhorn at ZDNet posted an interesting analysis of Google's Chrome OS announcement. The basic premise is that Google as a cloud information provider can subsidize a Netbook since it will get it back in cloud service revenue and a higher intangible value to its core business of collecting and characterizing customer behavior.

This is much like the telecom business or the game console business, and I have heard that same story from the reps at Samsung, Nokia, Asus, and Sony. It is just that Google has a big head start in the intangible value department.

But buried in this article is the core observation in my mind why the boxed world of software is transitioning to the cloud: security and cost.

"The problem is that Netbooks are cheap and, while they will gain in power they will stay cheap. I spent $270 on my HP Mini and that’s about right.

Microsoft has reportedly cut the price of Windows to $3 to capture Netbook OEMs, and it’s offering a cut-rate price on Office, too.

But when you consider the $50/year price to license an anti-viral, the $30/year to license a malware program and the additional $30/year you need for a registry cleaner, the software price of a Netbook gets completely out of line with its hardware cost."

This is the same observation that can be used for any boxed software. The cost of the underlying hardware platform has shrunk in the past 20 years, but the software cost hasn't kept pace. 20 years ago a workstation cost $75k so a $75k piece of software was reasonable. The cost of a workstation is now $2k, but the software is still $75k. The productivity improvement that I need to get from the software to justify the cost is too high and thus that type of cost can only be carried by a business model that has significant intangible value. And that value isn't present in the consumer and/or SMB market.

The smart phone started this trend and the netbook will accelerate it: the bulk of the market will be delivered services through subsidized hardware and software and it is the service providers that call the shots. Google, Amazon, Microsoft, Apple, Sony are already transitioning into these roles and since they have a connection with the bottom of the market pyramid, they will attract so much money that they will quickly roll over the Adobes, Oracles and SAPs of the world.

Many independent software vendors will clamor on the infrastructures of Google, Amazon, and Apple, and intangible value will be created. The enterprise market, of all markets, can't be isolated from the bulk of the money and they will need to adapt to the system where the information resides: and that will be the cloud.