IBM CIO Study: BI and Analytics are #1 Priority for 2012/2013

“Running a company is an endless quest to find out things you don’t know“

– Jeff Immelt, CEO GE

What will 2012 bring?  Recently, I attended the CIO Executive Leadership Summit in Greenwich, Connecticut. I was particularly intrigued by the presentation by the new CIO of IBM, Jeanette Horan where she presented the projects she was tackling and how IBM is thinking about business analytics.

IBM is making a bet that “true leaders” will develop the capabilities required for making good and timely decisions in unpredictable and stressful environments.

IBM is adapting to this new data analytics reality by a rapid-fire acquisition strategy:  Cognos,  Netezza, SPSS, ILog, CoreMetrics, Algorithmics, OpenPages, Clarity Systems, Emptoris, DemandTec (for retail).  IBM also has other information management assets like Watson, DB2 etc.  They are building a formidable capability around the value chain: “Raw Data -> Aggregate Data -> Intelligence ->Insight -> Decisions” . They see this as a $20Bln opportunity.

Some stats about IBM. It has more than 426,000 employees, with nearly 100,000 contractors. Operations in 170 countries divided into five integrated operating teams.  They have over 100K IBMers  in India.  IBM has an interesting mix of mobile vs. traditional employees. 20% road warriors or mobile employees; 12% work-at-home employees and 6% based entirely at client sites.

Horan presented some slides about IBM CIO study based on face to face conversations with 3,018 CIOs, spanning 71 countries and 18 industries.

The chart that got my attention was the aggregated CIO priorities.  It is pretty clear that Business Intelligence and Analytics is becoming a top initiative out there.  I am seeing the same thing in my discussions with CIOs. 2012 budgets have a lot of initiatives that are BI, Analytics and Data related.

BI and Analytics are in demand because the data volume growth is stunning… 1.8 ZB will be created and replicated in 2011.  The drivers are very strong:

• Up 9x in the last five years
• More than 90% of this data is unstructured
• Enterprises have some liability for 80% of this data
• Enterprises will spend $4T on managing data in 2011

IBM’s Big Data Strategy

To satisfy the market demand, IBM’s Big Data platform includes two major technologies:

1. IBM InfoSphere BigInsights and
2. IBM InfoSphere Streams.

IBM InfoSphere BigInsights is an analysis platform powered by Hadoop. The product is a bundled package of open source products with specific technologies from IBM that enables easy deployment, development, and monitoring of Hadoop. BigInsights integrates with IBM’s data warehouse solutions (DB2 and Netezza), using   modules that allow BigInsights to read and write from those products while running data processing jobs. This also includes a set of DB2 user-defined functions that enable invocation of Hadoop jobs through SQL, and easy exchange of data between the Hadoop File System and DB2 tables.

In contrast, IBM has developed InfoSphere Streams for real-time analytic processing to provide raw data pre-processing, feature extraction, data representation, filtering, aggregation, data (event) correlation, and anomaly detection and prediction.  While similar to complex event processing (CEP) systems, InfoSphere Stream enable continuous analysis of massive volumes of streaming data (both structured and unstructured) with sub-millisecond response times.  

• 

IBM is betting that data volume growth is going to spark a new wave of investment in the three legged analytics stool – people, process and technology.

Real-time analytics systems like Watson are now becoming powerful enough, and subtle enough, to help us turn noise into signals.  These systems can now chew through billions of bits of data, analyze them via self-learning algorithms, and package the insights for immediate use.

However the Big Data and Analytics hype scares me a little.  I don’t see a lot of corporations or IT organizations preparing or putting in place the necessary foundational elements – people, processes or decisions — to deal properly with a data-centric business.  Every senior manager I talk to about data quality or integration says… we have this issue covered by our SAP, Oracle Exadata or IBM Neteeza box  or some other XYZ project.

Implementing a BI tool from SAP or Oracle is not going to make the underlying data management issues magically disappear. Or spending a boatload of money on a data warehouse project doesn’t mean that data quality, standardization, matching and consolidation, lineage etc are taken care of.

There needs to be a lot more attention to the data fundamentals:

• How to understand the context (what process is the data supporting? First make sure that you have the full picture)
• How to simplify the process to simplify and make the data more manageable (there’s more data than capability to manage)
• How to integrate the data (across channels, applications and devices)
• How to improve quality of data (spot issues and gaps by Profile, Cleanse, Enrich, Match, Scorecard techniques )
• How to leverage the data (information about consumers, markets, opportunities)
• How to store the data (private cloud, public cloud, dedicated storage cloud)
• How to retrieve and visualize the data (especially from mobile devices)

Another key thing that is getting lost in the hype around BI and Analytics is the need for business process analysis and simplification. The smart organizations tend to identify all the steps in the process chain, eliminating whenever possible those steps that do not create value. Providing a lot of information in an imperfect business process with a lot of unnecessary steps or handoffs is not going to add much value.

Non-value adding work (process steps) generates a lot of useless data that is managed, integrated, stored, analyzed and retrieved.  Simplification is key. Big Data will help in processing but it will be “garbage in and garbage out” if we are not careful.

Simplifying key internal processes is probably a critical first step in BI and data management.  It might even result in better data quality as the “exhaust data or data scraps” gets eliminated.

Unfortunately few firms are getting this memo.  Management is not interested in the gory details. So get ready for some big blowups in the next few years as companies that don’t pay attention to the data context fundamentals pay the price.

Problems will manifest themselves in different ways: confidence in analytics systems declining, no single version of truth, and lot of extra time spent reconciling data.

Finally bringing it back to IBM.  IBM is investing heavily in  Analytics for internal use. They have an analytics-as-a-service project underway called Blue Insight.   It gathers information from nearly 100 different information warehouses and data stores.  The process areas where they are spending the money to drive better Business Enabling analytics around….

Notes and References

1)      The IBM CIOs Global Study:  http://www-935.ibm.com/services/c-suite/cio/study.html

2)      While the elimination of extra steps in a business process or elimination of  waste (lean thinking) may seem like a simple and clear subject it’s seldom practiced in IT organizations.

3) IBM Analytics Cloud — Blue Insight details  http://www-03.ibm.com/press/us/en/pressrelease/28823.wss

4)     Size of the BI and Analytics Market  

Related articles

• IBM takes its BI capabilities into the data center (networkworld.com)
• Oracle’s Analytics-as-a-Service Strategy: Exalytics, Exalogic and Exadata (practicalanalytics.wordpress.com)
• The Age of Big Data, New York Times, Feb 12, 2012
• Organizing for BI, Analytics and Big Data (practicalanalytics.wordpress.com)
• IDC: Analytics a $51B business by 2016 thanks to big data