Governmental IT: Analytics is not a dirty word

Over at Smart Data Collective, Bill Cooper wrote a wonderful article on deep analytics. In particular, I liked his assessment on the resistance expressed by customers that can't see the forest for the trees.

I’ve watched many government agencies balk at the idea of data mining and complex analytics. They are concerned about switching to a new data architecture and the potential risks involved in implementing a new solution or making a change in methodology.

Having been there, I do understand their concerns, but fear of change is what’s holding government agencies back from being able to fully leverage the data that already exists to effect change at the local, regional, state and national levels. Analytics are the key to lowering costs, increasing revenue and streamlining government programs.

In my own government experience and now, looking at it from the other side, I have come to believe that government clients need to think about data the way the world’s top corporations do. Like all federal agencies, these companies already had huge repositories of data that were never analyzed – never used to support decisions, plan strategies or take immediate actions. Once they began to treat that data as a corporate asset, they started to see real results. The best part is that leveraging these mountains of data does not require a "rip and replace" approach. Inserting a data warehousing/data mining or complex analytics capability into a SOA or cloud computing environment can be very low risk and even elegant in its implementation. The potential rewards are immense!

That’s what’s needed in the government sector. We need to view analytics not as a dirty word but as a secret weapon against fraud and other challenges impacting all areas of the government sector.

I am a big believer that the future of the cloud consists of federated systems for the simple reason that large data sets are captive to their storage devices. Federated systems makes service oriented architectures (SOA) a natural architecture pattern to collate information. The fact that Google Gears and Microsoft Azure exhibit SOA at different levels of abstraction is clear evidence of the power of SOA. Add coarse grain SOAs to these fine-grained patterns and you can support federation and scale internal IT systems even if the core runs in Gears or Azure.