Skip to content

Posts by Ravi Kalakota

18
Mar

Data-as-a-Service (DaaS)


datamartproliferation

If the analytics team wrestles with getting access to data, how timely are the insights?

To address the question…Global CIO are shifting their strategy — “need to build data-as-a-service offering for my data” to enable the analytics users in the organization.   The more advanced CIOs are asking – “how should I build data science capabilities as a shared foundation service?”

The CIO challenge is not trivial. Successful organizations today operate within application and data eco-systems which extend across front-to-back functions (sales & marketing all the way to fulfillment and service) and well beyond their own boundaries. They must connect digitally to their suppliers, partners, distributors, resellers, regulators and customers. Each of these have their “data fabrics” and applications which were never designed to connect, so with all the data-as-a-service and big data rhetoric, the application development community being asked to “work magic” in bringing them together.

Underutilization and the complexity of managing growing data sprawl is not new. But the urgency to address this is increasing dramatically during the last several years. Data-as-a-Service (DaaS) is seen as a big opportunity in  improving IT efficiency and performance through centralization of resources. DaaS strategies have increased dramatically  in the last few years with the maturation of technologies such as data virtualization, data integration, MDM,  SOA, BPM  and Platform-as-a-service.

The questions which are accelerating the Data-as-a-Service (DaaS) trend:  How to deliver the right data to the right place at the right time? How to “virtualize” the data often trapped inside applications?  How to support changing business requirements (analytics, reporting, and performance management) in spite of ever changing data volumes and complexity.

Read more »

17
Jan

Data Management, AML, and KYC Analytics


Financial Services value chainTo roadmap Wall Street priorities for 2014, we have been having an interesting set of meetings recently with MDs and leading architects in global banks and investment services firms.

It is clear that banks are devoting more resources to Know Your Customers (KYC),  Anti-Money Laundering (AML), fraud detection and prevention, Office of Foreign Assets Control (OFAC) compliance.

Got the scoop on analytics projects they are investing in — Anti-Money Laundering (AML) monitoring, Risk and Regulatory Management, trade surveillance and Know Your Customer (KYC) analytics.

To enable AML and KYC initiatives…the big foundational investments are around:

1)     Strengthening the Golden Sources – Security Master, Account Master and Customer Master.

2)     Various enterprise data management initiatives – Data Quality, Data Lineage, Data Lifecycle Management, Data Maturity and Enterprise Architecture procedures.

3) Regulatory reporting improvements via next generation Enterprise Datawarehouses (EDW) — Reporting on top of EDW addresses the core problems faced by Finance, Risk and Compliance when these functions extract their own feeds of data from the product systems through which the business is conducted and use differing platforms of associated reference data in support of their reporting processes. Lot of current investments are in the areas of Finance EDW which delivers common pool of contracts, positions and balances, organized on an enterprise wide basis and completed by anointed “gold” sources of reference data which ensure consistency and integration of information.

Crawl, walk, Run seems to be the execution game-plan as the data complexity is pretty horrendous. Take for instance, Citi alone….has approximately 200 million accounts and business in 160+ countries and jurisdictions.

The type of data challenges global banks like Citigroup and JP MorganChase are wrestling with include: Read more »

15
Jan

Big Data Company Shakeout?


hype cycleBig Data is the latest “next big thing” transforming all areas of business, but amid the hype, there remains confusion about what it all means and how to create business value.

Usually when there is so much hype…there is an inevitable boom-bust-boom cycle. Hence my question:  Is the Big Data shakeout inevitable?

Are we in a big data tech bubble? If you are an enterprise customer, how do you prepare for this? What strategies do you adopt to take advantage of the situation? Can you move from lab experiments to production deployments with confidence?

The Case of Drawn to Scale

Drawn to Scale, the four year-old startup behind Spire, shut down recently. Co-founder and CEO Bradford Stephens announced the news in a blog post. Drawn to Scale raised .93M in seed funding.

Spire is a real-time database solution for HBase that lets data scientists query Hadoop clusters using SQL. According to Stephens, the system has been by deployed by American Express, Orange Flurry, and four other companies.

Drawn to Scale showed that its technology was viable in enterprise environments and established a “presence against  competitors who raised 10-100x more cash,” but even that wasn’t enough to save the startup from its financial woes.

As Hadoop evolves and different layers of the data analytics stack get commoditized, specialized vendors like Drawn to Scale will have problems surviving.   SQL-on-Hadoop was a unique feature set…but over time it has become a must-have feature, that is becoming embedded in the stack – e.g., Impala in Cloudera CDH stack.  As a result, firms like Drawn to Scale once unique functionality becomes difficult to monetize.

Startup to Viable Ventures

The Big Data ecosystem is exploding with exciting start-ups, new divisions and new initiatives from established vendors.  Everyone wants to be the vendor/platform of choice in assisting firms deal with the data deluge (Data growth curve: Terabytes -> Petabytes -> Exabytes -> Zettabytes -> Yottabytes -> Brontobytes -> Geopbytes), translate data to information to insight, etc.

In both U.S and Europe, several billion dollars of venture money has been invested in the past three years alone in over 300+ firms.  Firms like Splunk had spectacular IPOs. Others like Cloudera and MapR have raised gobs of money. In the MongoDB space alone – a small market of less than 100M total revenue right now, over $2 Billion is said to have been invested in the past few years.

Read more »

11
Dec

20 must read Infograhics on Big Data


Big Data emphasizes the exponential growth of data volumes worldwide (collectively, >2.5 Exabytes/ day).

Big Data incorporate the following key tenets: diversification, low latency, and ubiquity. In parallel, the emerging field of data science introduces new terms including, predictive modeling, machine learning, parallelized and in-database algorithms, Map Reduce, and data monetization.

A variety of infographics have been published around Big Data, Data Scientists.  Here is a compendium of some very interesting ones.

The Real World of Big Data  (Click image to see a larger version and article)

Real World of Big Data

Big Data Big OpportunityBig Data Big Opportunity

A Data Scientist StudyA Data Scientist Study

22
Oct

Sizing “Mobile + Social” Big Data Stats


“Welcome to the Internet of Customers. Behind every app, every device, and every connection, is a customer. Billions of them. And each and every one is speeding toward the future.” Salesforce.com   

Mobile and social are major data exhaust producers. Mining this data is the new frontier.

Why…  Did you know that every 60 SECONDS, a tidal wave of unstructured data is being produced, consumed and archived via mobile devices.  As you read this ask yourself: what does this mean?

Companies are racing to comeup with new ways to leverage “next best action/offer” analytics in a world where customer experience is getting more complex.  Take retail for instance. In a multi-channel and multi-device world, as consumers move across channels, new techniques are needed to capture and increase conversion rate. (Conversion rate is the percentage of people who come to your website and take desired actions, such as purchasing something or requesting more information.)

Imagine this scenario…. let’s say a friend tweets about a new 60 inch Samsung smart TV they bought at Best Buy. You read the tweet, but click on the URL on the mobile device and check it out.  Even though that was the last click, what made the transaction happen was a satisfied friend posting a recommendation via social media and retrieved on a mobile device.  The ability to convert the visitor requires analytics… where they came from, what caused them to come to the site, what offer to present, etc.

Social technology adoption and usage by consumers is no longer an early adopter market — it’s a mainstream activity.  Mobile is accelerating this trend. All this means a “new customer interaction” model powered by big data is emerging.

Why is big data analytics a good lens for creating value around social:

  • New data is coming across multiple dimensions – demographic, geographic, psychographic, behavioral, socialgraphics
  • Business decisions approach real-time. Time available to capture data is decreasing.  Analysis of increasing data volumes have to become faster. Operational excellence requires immediate action.  Real-time capture and action is where the state of the art is.
  • Coupled with mobile and cloud, it means the emergence of a new Customer Interaction Model for corporations

All this data growth and value creation trends imply that data management, Big Data and real-time analytics is  a big focus in social and mobile data going forward.  Clearly a new style of IT is emerging (see this figure from HP Analyst Briefing which conveys the computing transformation message quite well).

Read more »

2
Oct

Enterprise Data Architecture and Big Data


Image

“Through 2015, more than 85 percent of
Fortune 500 organizations will fail to effectively exploit big data for competitive advantage” - Gartner BI Summit.

It doesn’t take genius to recognize that there is an increasing demand for information to improve shareholder value and gain competitive advantage by leveraging information, data and analytics as a strategic enterprise asset. The question is no longer about the importance of data but when, how, and where to leverage the asset.  Read more »

20
Aug

Innovation and Big Data: A Roadmap


The bleeding edge of insight innovation is around next generation digital consumer experience.  Consumer behaviors are rapidly evolving….always connected, always sharing, always aware. Obviously new technology like Big Data drives and transforms  consumer behavior and empowerment.

With the influx of money, attention and entrepreneurial energy, there is a massive amount of innovation taking place to solve data centric problems (such as the high cost of collecting,  cleaning, curating, analyzing, maintaining, predicting) in new ways.  

There are two distinct patterns in data-centric  innovation:

  • Disruptive innovation like predictive search which brings a very different value proposition to tasks like discover, engage, explore and buy and/or creates new markets!!
  • Sustaining innovation like mobile dashboards,   visualization  or data supply chain management which improves self service and performance of existing products and services.

With either pattern the managerial challenge is moving from big picture strategy to day-to-day execution.  Execution of big data or data-driven decision making requires a multi-year evolving roadmap around toolset, skillset, dataset, and mindset. 

Airline loyalty programs are a great example of multi-year evolving competitive roadmaps. Let’s look at BA’s Know Me project.

British Airways “Know Me” Project

British Airways (BA) has focused on competitiveness via customer insight. It has petabytes of customer information from its Executive Club loyalty program and its website. BA decided to put customer big data to work in its Know Me program. The goal of the program is to understand customers better than any other airline, and leverage customer insight accumulated across billions of touch points to work.

BA’s Know Me program  is using data and applying it to customer decision points in following ways:

  • Personal recognition—This involves recognizing customers for being loyal to BA, and expressing appreciation with targeted benefits and recognition activities
  • Personalization – based on irregular disruptions like being stuck on a freeway due to an accident – A pre-emptive text message… We are sorry that you are missing your flight departure to Chicago. Would you like a seat on the next one at 5:15PM.  Please reply Yes or No.
  • Service excellence and recovery—BA will track the service it provides to its customers and aim to keep it at a high level. Given air travel constant problems and disruptions, BA wants to understand what problems its customers experience, and do its best to recover a positive overall result
  • Offers that inspire and motivate—BA’s best customers are business travelers who don’t have time for irrelevant offers, so Know Me program analyzes customer data to construct relevant and targeted “next best offers” for their consideration.

The information to support these objectives is integrated across a variety of systems, and applied in real-time customer interactions at check-in locations and lounges. Even on BA planes, service personnel have iPads that display customer situations and authorized offers. Some aspects of the Know Me program have already been rolled out, while others are still under development.

The Need for New Data Roadmaps

New IT paradigms (cloud resident apps, mobile apps, multi-channel, always-on etc.) are creating more and more complex integration landscapes with live, “right-now” and real-time data. With data increasingly critical to business strategy, the problems of poor quality data,  fragmentation, and lack of lineage are also taking center stage.

The big change taking place in the application landscape: application owners of the past expected to own their data. However, applications of the future will leverage data – a profound change that is driving the data-centric enterprise.  The applications of the future need one “logical” place to go that provides the business view of the data to enable agile assembly.

Established and startup vendors are racing to fill this new information management void.  The establish vendors are expanding on this current enterprise footprint by adding more features and capabilities. For example, the Oracle BI stack (hardware – databases – platform – prebuilt content) illustrates the data landscape changes taking place from hardware to mobile BI apps.  Similar stack evolution is being followed by SAP AG, IBM, Teradata and others.  The startup vendors typically are building around disruptive technology or niche point solutions.

To enable this future of information management,  there are three clusters of “parallel” innovation: (1) technology/infrastructure centric; (2) business/problem centric; and (3) Organizational innovation.

Data Infrastructure Innovation

  • Data sources and integration — Where does the raw data come from?
  • Data aggregation and virtualization- Where it stored and how is it retrieved?
  • Clean high quality data — How does the raw data get processed in order to be useful?

Even in the technology/infrastructure centric side there are multiple paths of disruptive innovation that are taking along different technology stacks shown below.  

Read more »

2
Jul

Enabling SoLoMoMe + Omni-channel Analytics


At the Analytics Executive Forum, I facilitated a session on Omni-channel analytics. It struck me how every leading consumer facing firm seems convinced that mobile is becoming the dominant B2C interaction channel.  Mobile is the gateway to insight based marketing and the “always addressable customer”….

Insight-based interactions –  The company knows who you are, what you prefer, and communicates with relevant, timely messages, using the power of analytical intelligence to detect patterns, decode strands of information and create meaningful offers and value.

The “always addressable customer.” This is a consumer who fits the bill on three fronts simultaneously: (1)

  • Owns and personally uses at least three connected devices; (2)

Goes online multiple times throughout the day;  (3) 

  • Goes online from at least three different physical locations

The opposite of insight-based is “spray-and-pray” marketing - The company has very limited knowledge about who you are, forgets what you prefer, and tries to reach you with off-target communications that alienate you – based on fragmented data, poor data quality and  inadequate integration, resulting in confusing, chaotic interactions.  A good example: “I have 2 million frequent flyer miles with your airline and still do not get any recognition, respect or value from this loyalty.”

As companies architect new insight based mobile use cases I suggest that they look at what is coming next. With IOS 7, Apple is delivering several new features – Passbook, Beacon.

Retailers, banks and other customer facing firms/brands better pay attention. 100+ million iPhones are automatically getting this feature with the new OS upgrade making this a mega-disruptor in the coveted target segment everyone is chasing. Read more »

19
Jun

Organizing for BI, Analytics and Big Data: CoE, Federated or Departmental


Consider this…eBay’s “Singularity” Teradata warehouse exceeds 40 petabytes. According to eBay, the company’s data volumes are 50+ terabytes per day in new incremental data, processing 50+ petabytes
and tens of millions of queries per day, with 99.98% availability and more than 50 petabytes of online storage.

Data is valuable. Data is plentiful. Data is complex. Data is in flux. Data is fast moving. Capturing and managing data is challenging.

So, if you are a senior leader in a Fortune 2000 company.  How do you structure your group to deliver effective BI, Analytics or Big Data projects? Do you have the right structure,  toolset, dataset, skillset and mindset for analytics and Big Data?

Organizing for effective BI, Analytics and Big Data is becoming a hot topic in corporations.  In 2012, business users are exerting  significant influence over BI, Analytics and Big Data decisions, often choosing analytics and visualization platforms and products in addition to/as alternatives to traditional BI platform (reporting and visualization tools).

Read more »

4
Jun

Omni-channel Conversion Optimization with Social Marketing/Media Analytics


Brooks Brothers is investing in tools & testing to improve the online experience – and ↑sales. In a test involving one product category page: men’s shirts. The retailer using Bazaarvoice Ratings & Reviews software, used customer reviews to sort items on the product page. Items with five ♥♥♥♥♥- the highest rating – appeared on the top of the page. The result:  a 9% lift in conversions   [Adobe Digital Marketing Symposium]

Are you ready to anticipate and influence your audience in a whole new way? Value migration from traditional marketing to 24×7 digital marketing is happening in leading firms.  Real-time marketing and conversion is now becoming possible.

Read more »

Follow

Get every new post delivered to your Inbox.

Join 263 other followers

%d bloggers like this: