Data Scientist Infographic & Managed Analytics

The exploding demand for analytics professionals has exceeded all expectations, and is driven by the Big Data tidal wave.  Big data is a term commonly applied to large data sets where volume, variety, velocity, or multi-structured data complexity are beyond the ability of commonly used software tools to efficiently capture, manage, and process.

To get value from big data, ‘quants’ or data scientists are becoming analytic innovators who create tremendous business value within an organization, quickly exploring and uncovering game-changing insights from vast volumes of data, as opposed to merely accessing transactional data for operational reporting.

This EMC infographic summarizing their Data Scientist study supports my hypothesis – Data is becoming new oil and we need a new category of professionals to handle the downstream and upstream aspects of drilling, refining and distribution. Data is one of the most valuable assets within an organization. With business process automation, the amount of data  being generated, stored and analyzed by organizations is exploding.

Following up on our previous blog post – Are you one of these — Data Scientist, Analytics Guru, Math Geek or Quant Jock? — I am convinced that future jobs are going to be centered around “Raw Data -> Aggregate Data -> Intelligence ->Insight -> Decisions”  data chain.   We are simply industrializing the chain as machines/automation takes over the lower end of the spectrum. Also Web 2.0 and Social Media are creating an interesting data feedback loop – users contribute to the products they use via likes, comments, etc.

CIOs are faced with the daunting task of unlocking the value of their data efficiently in the time-frame required to make accurate decisions. To support the CIOs, companies like IBM are attempting to become a one-stop shop by a rapid-fire $14 Bln plus acquisition strategy:  Cognos,  Netezza, SPSS,  ILog, Solid, CoreMetrics, Algorithmics, Unica, Datacap, OpenPages, Clarity Systems, Emptoris, DemandTec (for retail).  IBM also has other information management assets like Ascential, Filenet, Watson, DB2 etc.  They are building a formidable ecosystem around data. They see this as a $20Bln per year opportunity in managing the data, understanding the data and then acting on the data.

The reliance on analytics is changing in organizations.  It was used to generate   reports and guide decision making in financial management and supply chain management. Now the emphasis is drive predictive strategies, and to guide activities in selling, marketing and operations. Ad-hoc spreadsheets are giving way to sophisticated real-time insight engines. We saw this transition in finance in the past decade as algorithmic and high frequency trading changed the landscape.

Another interesting trend I see unfolding due to shortage of analytics talent in companies is the growth in outsourced analytics.   Outsourced analytic providers serve many industries, including retail, telecommunications, healthcare and others. They provide clients with domain expertise in predictive models for pricing and customer segmentation. Since clients are looking for faster time-to-insight, outsourcers are hiring and training talent to build “data-as-a-service” platforms that that will scale better with lower cost of ownership to meet their clients’ service-level agreements.

The Age of Big Data, New York Times, Feb 12, 2012