Skip to content

Apps+Infrastructure 101

A hacker compromises a credit card account and manages to make a few purchase, plenty of damage has already been done, even if that account is cut off within minutes.

What apps and infrastructure do you put in place to prevent this?

Want to model the big picture. Need a visual drag-and-drop approach that allows data analysts (or any designated user) throughout an organization to work with large data sets, develop and refine models, and collaborate at scale without having to code.

What apps and infrastructure do you put in place to enable this?

  • Monetizing Analytics and Data is #1 priority in most firms.
  • Making analytics operational… Planning and architecting data-driven decision-making is a growing strategic imperative
  • Laying the groundwork….What apps do you invest in?
  • What platforms and infrastructure do you implement?

Recent research indicates that many CEOs lack clarity about the CDO (chief data/analytics officer) role and what it takes to execute enterprise level analytics projects.

The result: false starts, failed digital initiatives and ceding valuable time to competitors who already have a better understanding.

The complexity of apps and infrastructure around Analytics Projects is mind-boggling… hence the challenge in translating grand visions to execution.

  • What are the tools that turn data into intelligence?  Right tools for descriptive, prescriptive, predictive analytics,  exploratory vs. confirmatory analysis?
  • What about the data infrastructure?   Between 70% and 80% of the time spent on an enterprise analytics project is consumed by preparing the data. Most people try to short-cut this. Sadly…it’s “Garbage in, Garbage out” if data infrastructure is not robust. Be sure to carefully QA data feeds, sources to ensure its integrity.
  • What is the right platform? The enterprise analytics market is in the middle of an accelerated transformation from traditional BI systems (focus on measurement & reporting) to those that also support analysis, prediction, forecasting, simulation and optimization.

How to future proof investment decisions? New sub-categories are constantly emerging such as social media analytics – Analytics that listen, measure and analyze social media performance to grow the business, enhance reputation, improve customer experience. Social media analytics is a hot category as net promoter/influencer driven word-of-mouth and peer-to-peer interaction is more powerful than traditional advertising.

Essentially a lot of decisions have to be made about core feature/functionality – analytics foundation building blocks – from which everything is derived.  The challenge always in very project is how to go from Raw Building Blocks ->Modular Application Components -> Platforms


Enable Better Targeting

Turn customer-related data from inside and outside your organization — real-time, batch,  structured, and unstructured — into powerful, individuated marketing programs and actions.

  • Engage in 1:1 conversations with individual customers, across any channel.
  • Driving customer loyalty through improving recommendations.
  • New technologies for connecting with customers… the convergence of marketing, advertising, sales, and public relations

Engagement Analytics and Personalization: Delivering targeted content based on user behaviors and preferences. Display relevant content in the right context to achieve desired results. And the reason organizations are increasingly turning to it is that it improves responses, boosts conversions, and delivers a superior customer experience.

Relevant technologies – segmentation management, ad serving, mobile content targeting, email campaign management.

A/B and multivariate tests:   A/B testing is a way to test design changes and determine which ones produce positive results. A/B testing allows you to show visitors two versions of the same page and let them determine the winner.  It is a method to validate that any new design or change to an element on the webpage is improving your conversion rate before you make that change to your site code. A/B Testing takes the guesswork out of website optimization and enables quantitative data-backed decisions that shift business conversations from “we think” to “we know.” By measuring the impact that changes have on  metrics such as sign-ups, downloads, purchases, you can ensure that every change produces positive results.

Relevant technologies – A/B testing, Reporting, Web Analytics, Social Analytics, Search Engine Optimization, Polling and Feedback

Enable Better Information Management

Today’s explosion of digital information requires firms to analyze new information faster and make timely and well founded decisions. Information management addresses that need not only the right policies, skills and processes but also a solid foundation in technology. It is a holistic approach to managing, improving and leveraging information to increase an organization’s confidence in decisions made–within big data and analytics, and within operational business processes.

  • Master Data Management – Golden Source data about customers, suppliers, partners, products, materials, accounts and other critical “entities,” that is commonly stored and replicated across IT systems
  • Data Quality – convert enterprise data into trusted data;
  • Data Profiling & Data Lineage - silo’d disconnected applications and systems drive inefficiencies and promote version control problems

Enable Better Information Delivery

  • Pace of presentation (interaction, visualization) is accelerating
  • Run the Business (Organize the data to do something specific)
  • Change the Business (Take data as-is to figure out what it can do)
  • Zero in on what matters

Reporting: Provides the ability to create highly formatted, print-ready and interactive reports, with or without parameters.

Dashboards: A style of reporting that graphically depicts performances measures. Includes the ability to publish multi-object, linked reports and parameters with intuitive and interactive displays; dashboards often employ visualization components such as gauges, sliders, checkboxes and maps, and are often used to show the actual value of the measure compared to a goal or target value. Dashboards can represent operational or strategic information.

Ad hoc report/query: Enables users to ask their own questions of the data, without relying on IT to create a report. In particular, the tools must have a reusable semantic layer to enable users to navigate available data sources, predefined metrics, hierarchies and so on.

Mobile Apps and Alerts: Enables organizations to develop and deliver content to mobile devices in a publishing and/or interactive mode, and takes advantage of mobile devices’ native capabilities, such as touchscreen, camera, location awareness and natural-language query. See Mobile BI for more

Microsoft Office integration: Often Microsoft Office (particularly Excel) acts as the reporting or analytics tool.  Various BI tools provide integration with Microsoft Office, including support for native document and presentation formats, formulas, charts, data “refreshes” and pivot tables. Advanced integration includes cell locking and write-back.


Enable Better Analysis

  • What-if real-time analysis enabled by near real-time DW refresh
  • Smarter KPIs – Past, Present and Future
  • Summary level KPIs with drill down capabilities to identify exceptions

Interactive visualization: Enables the exploration of data via the manipulation of chart images, with the color, brightness, size, shape and motion of visual objects representing aspects of the dataset being analyzed. This includes an array of visualization options that go beyond those of pie, bar and line charts, including heat and tree maps, geographic maps, scatter plots and other special-purpose visuals. These tools enable users to analyze the data by interacting directly with a visual representation of it.

Search-based data discovery: Applies a search index to structured and unstructured data sources and maps them into a classification structure of dimensions and measures that users can easily navigate and explore using a search interface. This is not the ability to search for reports and metadata objects. This would be a basic feature of a BI platform.

Geospatial and location intelligence: Specialized analytics and visualizations that provide a geographic, spatial and time context. Enables the ability to depict physical features and geographically referenced data and relationships by combining geographic and location-related data from a variety of data sources, including aerial maps, GISs and consumer demographics, with enterprise and other data. Basic relationships are displayed by overlaying data on interactive maps. More advanced capabilities support specialized geospatial algorithms (for example, for distance and route calculations), as well as layering of geospatial data on to custom base maps, markers, heat maps and temporal maps, supporting clustering, geofencing and 3D visualizations.

Embedded advanced analytics: Enables users to leverage a statistical functions library embedded in a BI server. Included are the abilities to consume common analytics methods such as Predictive Model Markup Language (PMML) and R-based models in the metadata layer and/or in a report object or analysis to create advanced analytic visualizations (of correlations or clusters in a dataset, for example). Also included are forecasting algorithms and the ability to conduct “what if?” analysis.

Online analytical processing (OLAP): Enables users to analyze data with fast query and calculation performance, enabling a style of analysis known as “slicing and dicing.” Users are able to navigate multidimensional drill paths. They also have the ability to write-back values to a database for planning and “what if?” modeling. This capability could span a variety of data architectures (such as relational, multidimensional or hybrid) and storage architectures (such as disk-based or in-memory).

Enable Better Integration and Data Logistics

  • Any data, Any Source, Any Format
  • On Premise, On the Cloud, On Mobile
  • Unified Data Architecture – Any User, Any Data, Any Analysis

BI infrastructure and administration: Enables all tools in the platform to use the same security, metadata, administration, object model and query engine, and scheduling and distribution engine. All tools should share the same look and feel. The platform should support multi-tenancy.

Metadata management: Tools for enabling users to leverage the same systems-of-record semantic model and metadata. They should provide a robust and centralized way for administrators to search, capture, store, reuse and publish metadata objects, such as dimensions, hierarchies, measures, performance metrics/key performance indicators (KPIs), and report layout objects, parameters and so on. Administrators should have the ability to promote a business-user-defined data mashup and metadata to the systems-of-record metadata.

Business user data mashup and modeling: Code-free, “drag and drop,” user-driven data combination of different sources and the creation of analytic models, such as user-defined measures, sets, groups and hierarchies. Advanced capabilities include semantic autodiscovery, intelligent joins, intelligent profiling, hierarchy generation, data lineage and data blending on varied data sources, including multistructured data.

Development tools: The platform should provide a set of programmatic and visual tools and a development workbench for building reports, dashboards, queries and analysis. It should enable scalable and personalized distribution, scheduling and alerts of BI and analytics content via email, to a portal and to mobile devices.

Embeddable analytics: Tools including a software developer’s kit with APIs for creating and modifying analytic content, visualizations and applications, embedding them into a business process, and/or an application or portal. These capabilities can reside outside the application, reusing the analytic infrastructure, but must be easily and seamlessly accessible from inside the application, without forcing users to switch between systems. The capabilities for integrating BI and analytics with the application architecture will enable users to choose where in the business process the analytics should be embedded.

 BigData Use Case - Data Logistics

Enable Better DataWarehousing and Virtualization

  • Drive an enterprise level single view of the business  - all steps of that preparation process, including data sourcing, validation, cleansing, organization, and hierarchy
  • Centralize, decentralize, virtualize… Deal with proliferation of data marts has resulted in fragmented data, higher costs and poor decisions
  • Data quality… Garbage in, garbage out. Be sure to carefully QA data feeds, sources to ensure its integrity.
  • Structured and unstructured data from any and all data sources—including databases, XML feeds, CSV output, EBCDIC, social media and more.

Datawarehousing:  Load once, use many times…enable cross functional analysis. Integrated data provides data consistency, lower costs and better decisions…only if implemented well.  Many datawarehouses projects tend to become white elephants and endup delivering only a fraction of the approved business case.

Data Virtualization:  Virtual datamarts and Virtual data stores are increasingly common consolidation style architecture  implementation today.  The target goal is lower cost of execution ($/query, $ / user), and agility by leveraging SOA principles such as abstraction, shared semantic models and data standards.


2014 Magic Quadrant – Vendors to Consider 

Choosing which platform, toolset for what purposes?

There are many considerations for analytics tool and platform selection.  Data types, business requirements, total cost of ownership, lock-in, as-is and to-be processes etc.  Also there are wide range of costs to consider. License fees are just one piece of the puzzle.

Bottomline… How can you most efficiently solve the business problem today? Don’t get into endless religious debates about vendors, tool capability and future gaps. A good team can deliver the target solution with almost any vendor product.

Gartner for their Magic Quadrant vendor analysis came up with a way to score vendors based on  17 categories of core feature/functionality. This MQ analysis  I think is extremely useful to any IT executive as they try to cut thru the vendor clutter and noise.

Gartner’s 2014 Magic Quadrant as a Reference for Vendor Landscape analysis.


Putting all together…. A related landscape figure that i found useful is from Forrester Research.  Just illustrates the complexity IT executives have to wade thru.

BI-Analytics Stack

Summary – The Evolution to Analytics Platform Standardization

Analytics apps  are an integral part of an organization’s fabric providing critical information to various departments. As the number of departmental applications, powered by different technologies, in an organization grow, the IT departments face a challenge to manage and maintain them. Ultimately, an organization will advance from having isolated  applications towards platform standardization.

The evolution tends to happen in 4 phases:

  • Many companies deploy Analytics (and BI) applications as departmental solutions, and in the process, have accumulated a large collection of disparate BI technologies as a result. Each distinct technology supported a specific user population and database, within a well-defined “island of analytics.” At first, these dept islands satisfied the initial needs of the business, but early success in departmental deployment sowed the seeds for new problems as the applications grew.
  • Successful applications always expand. The second phase of Analytics (and BI) is hallmarked by BI applications that have expanded to the point where they are no longer isolated islands. Instead, they overlap in user populations, data access, and analytic coverage. As a result, organizations are now faced with an untenable situation. The enterprise is getting conflicting versions of the truth through the multiple disparate BI systems, and there is no way to harmonize them without an extraordinary ongoing manual effort of synchronization. Equally problematic is the fact that business users are forced to use many different BI tools depending on what data they want.
  • The third phase of Analytics (and BI)  is one where a single platform delivers one version of the truth (golden source of data) to all people across the enterprise. It can access all of the data, administer all of the people, eliminate repetitive data access, reduce the administrative effort, and reduce the time to deploy new BI applications.
  • The fourth phase tends to be Analytics Outsourcing and CoE.  As the platform becomes standardized and there is continually less evolution to greater functionality, greater scalability, with seamless integration, and ever greater economies of scale.  Cost pressures and slowing of innovation will cause management to look for external vendors to run the platform cheaper.



Notes and References

1. Source: Gartner 2012 Worldwide Survey of More Than 2,300 CIOs Survey Shows CIOs are Using Technology to “Amplify” Enterprise.

Why Do BI and Analytics Projects Fail?

Executing a BI CoE

BI and Data CoE

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Note: HTML is allowed. Your email address will never be published.

Subscribe to comments


Get every new post delivered to your Inbox.

Join 259 other followers

%d bloggers like this: