Do you have the right toolset, dataset, skillset and mindset for analytics? Do you want to enable end users to get access to their data without having to go through intermediaries?
The challenge facing managers in every industry is not trivial… how do you effectively derive insights from the deluge of data? How do you structure and execute analytics programs (Infrastructure + Applications + Business Insights) with limited budgets?
The exploding demand for analytics professionals has exceeded all expectations, and is driven by the Big Data tidal wave. Big data is a term commonly applied to large data sets where volume, variety, velocity, or multi-structured data complexity are beyond the ability of commonly used software tools to efficiently capture, manage, and process.
To get value from big data, ‘quants’ or data scientists are becoming analytic innovators who create tremendous business value within an organization, quickly exploring and uncovering game-changing insights from vast volumes of data, as opposed to merely accessing transactional data for operational reporting.
This EMC infographic summarizing their Data Scientist study supports my hypothesis – Data is becoming new oil and we need a new category of professionals to handle the downstream and upstream aspects of drilling, refining and distribution. Data is one of the most valuable assets within an organization. With business process automation, the amount of data being generated, stored and analyzed by organizations is exploding.
Following up on our previous blog post – Are you one of these — Data Scientist, Analytics Guru, Math Geek or Quant Jock? – I am convinced that future jobs are going to be centered around “Raw Data -> Aggregate Data -> Intelligence ->Insight -> Decisions” data chain. We are simply industrializing the chain as machines/automation takes over the lower end of the spectrum. Also Web 2.0 and Social Media are creating an interesting data feedback loop – users contribute to the products they use via likes, comments, etc.
CIOs are faced with the daunting task of unlocking the value of their data efficiently in the time-frame required to make accurate decisions. To support the CIOs, companies like IBM are attempting to become a one-stop shop by a rapid-fire $14 Bln plus acquisition strategy: Cognos, Netezza, SPSS, ILog, Solid, CoreMetrics, Algorithmics, Unica, Datacap, OpenPages, Clarity Systems, Emptoris, DemandTec (for retail). IBM also has other information management assets like Ascential, Filenet, Watson, DB2 etc. They are building a formidable ecosystem around data. They see this as a $20Bln per year opportunity in managing the data, understanding the data and then acting on the data. Read more
The change in consumer behavior and expectations that e-commerce, mobile and social media are causing is hugely significant – big data and predictive analytics will separate brand/retail winners from losers. This won’t happen overnight but the transformation is for real.
Retail Industry makes up a sizable part of the world economy (6-7%) and covers a large ecosystem - E-commerce, Apparel, Department Stores, Discount Drugstores, Discount Retailers, Electronics, Home Improvement, Specialty Grocery, Specialty Retailers and Consumer Product Goods suppliers.
Retail is increasingly is looking like a barbell – a brand oriented cluster at the high-end, a very thin middle, and a price sensitive cluster at the low end. The consumerization of technology is putting more downward pricing pressure in an already competitive “middle” retail environment. The squeeze is coming from e-commerce and new “point, scan and analyze” technologies that give shoppers decision making tools — powerful pricing, promotion and product information, often in real-time. Applications in iPhones and Droid, like Red Laser can scan barcodes and provide immediate price, product and cross-retailer comparisons. They can even point you to the nearest retailer who can give you free shipping (total cost of purchase optimization). This will lead to further margin erosion for retailers that compete based on price (a sizable chunk of the market in the U.S, Europe and Asia).
Data analytics is not new for retailers. Point of sale transactional data obtained from bar-codes first appeared in 1970s. A pack of Wrigley’s chewing gum was the first item scanned using Universal Product Code (UPC) in a Marsh Supermarket in Troy, Ohio in 1974. Since then, retailers have been applying analytics to get even smarter.
More recent use cases of retail analytics include: Read more
The “Raw Data -> Aggregated Data -> Intelligence -> Insights -> Decisions” is a differentiating causal chain in business today. To service this “data->decision” chain a very large industry is emerging.
The Business Intelligence, Performance Management and Data Analytics is a large confusing software category with multiple sub-categories — mega-vendors (full stack, niche vendors, data discovery, visualization, data appliances, Open Source, Cloud – SaaS, Data Integration, Data Quality, Mobile BI, Services and Custom Analytics).
But the interest in BI and analytics is surging. Arnab Gupta, CEO of Opera states why analytics are taking center stage, “We live in a world where computers, not people, are in the driver’s seat. In banking, virtually 100% of the credit decisions are made by machines. In marketing, advanced algorithms determine messages, sales channels, and products for each consumer. Online, more and more volume is spurred by sophisticated recommender engines. At Amazon.com, 40% of business comes from its “other people like you bought…” program.” (Businessweek, September 29, 2009).
Here is a list of vendors who participate in this marketspace:
However, it took until 1980s when decision support systems (DSS) became popular and mid 1990s for BI started to emerge as an umbrella term to cover software-enabled innovations in performance management, planning, reporting, querying, analytics, online analytical processing, integration with operational systems, predictive analytics and related areas.
BI, Analytics [and Big Data] Market Sizing
“There are many methods for predicting the future. For example, you can read horoscopes, tea leaves, tarot cards, or crystal balls. Collectively, these methods are known as “nutty methods.” Or you can put well-researched facts into sophisticated computer models, more commonly referred to as “a complete waste of time.”
Scott Adams, The Dilbert Future
Are you clear on your objective? What is the most important value proposition that you want to achieve through BI and analytics enabled strategies?
- Reduction in operating expenses
- Increased profitability
- Improve growth, competitiveness and market position
- Customer acquisition, loyalty and retention
- Product development and differentiation
The mis-alignment between what C-suite wants and what IT is capable of delivering is quite extraordinary. Many CFOs, CEOs believe that IT is unable to deliver results where it counts: the top line and bottom line. At the same time, IT organizations spend an incredible amount of time, money and resources simply reporting the obvious data within their business processes and workflows. The data overload is making find the obvious in the increasing tidal wave of structured and unstructured data a full-time job. As organizations emerge from the deep recession of 2008, the competitive pressures are putting even greater demands on the decision-making, KPIs and performance management processes of organizations.
To stay competitive means making better decisions more quickly. It means accelerating the “raw data -> clean data -> information -> insight -> decision cycle.” It dictates widening the scope and scale of the data management domain, the analytic landscape and the technological infrastructure.