Data/BI/Analytics Evolution @ NetFlix

More data + Better models + More accurate metrics + Better approaches & architectures = Lots of room for improvement!

It’s amazing to watch how quickly the data engineering / analytics/ reporting/ modeling/ visualization toolset is evolving in the BI ecosytem.

There are clearly massive foundational shifts taking place around big data. I am not sure how large conventional Fortune 500 firms can innovate and keep up with what’s going on.  I have run into CIOs who have not heard of Hadoop in some cases.

It’s also fascinating to see how data-driven “bleeding” edge firms like NetFlix are pushing the envelope.  Netflix stats are amazing:  1/3+ Internet traffic (NA / peak);  100+ Million hours per day; 65+ Million members / 50+ countries; 500 Billion Events / Day.

NetFlix is clearly reinventing Television and targeting 90 million potential subs in the US market alone.  Binge-watching, cord-cutting are now part of our everyday lingo. What most people don’t realize is how data-driven Netflix is…. from “giving viewers what they want” to “leveraging data mining to boost subscriber base”.

Viewing -> Improved Personalization -> Better Experience is the virtuous circle.

Here is a glimpse at how their BI landscape has evolved in the past five years as they integrate 5 million to 6 million net adds for several years now.  The figures are from a presentation by Blake Irvine, Manager Data Science and Engineering.

BI tools @ NetFlix pre-Hadoop