Goodbye Data Swamp, Hello Data Lake
Data lakes have been around for eight years, and while the original concept was promising, many of the lakes have unfortunately turned into data swamps. This has prompted criticism of the concept itself.
Making use of the data exhaust
Many customer actions leave a digital imprint — a sort of data exhaust. Content consumption produces weblogs, purchases make transaction records, and smartphones generate coordinates.
The potential of the data exhaust is broadly acknowledged, but rarely realised. Gartner calls it “dark data”: information that organisations collect, but fail to use for other purposes. The problem with the data exhaust is that it’s not all that useful in its raw form.
Luckily, humans are habit-forming machines. The data exhaust contains billions of individual actions which, when viewed together, tell us something about the people behind those actions.
For example, if we know that someone regularly visits Workshop Coffee, we can reason our way to conclusions much more sophisticated than “they like coffee, sell them more coffee.” Since we know that Workshop Coffee is a high-end, indie coffee shop that charges £3 for a very small cup, we can start to make informed guesses about the attributes of a regular customer.
If we could see a broader view of this person’s actions — other stores they’ve visited, articles they’ve read, and businesses they’ve called — we could begin to piece together a much more complete, human-like profile, including personality type, emotional drivers, and behavioral patterns.
In a world of “big data,” it’s tempting to think that data is data is data. You simply want to aggregate as much of it as possible.
But data has its own value chain; its own sources of supply and demand. A human-like profile is distinct from a data exhaust in the same way that a BMW is distinct from unrefined metal and plastic — no matter how much you collect, you will never produce a BMW without specifically deciding to manufacture one.
A human-like profile is a rich set of structured data that describes a single person in holistic way, manufactured from the data exhaust created by that person’s actions.
Such customer profiling is extremely valuable not only for for gaining customer insight, but also for targeting and personalisation. So how can we create them?
Transforming billions of customer actions into human-like profiles is no small tasks, but smart machines can help. Here’s an example of the sort of transformation we’re looking for:
Data Exhaust for Jennifer
Called 555-482-9141 for 4.3 minutes
Browsed www.somesite.com/page124.html
Visited +40.689060 -74.044636
Purchased LEV-JN-SL-36-GN, £299
Human-like Profile for Jennifer
A sporty, high-income 20-something woman
who likes fashion, European travel, and airline loyalty programs.
To produce such customer profiles, a smart machine must consider each person’s data exhaust, consult external sources of knowledge like ontologies, business directories, and the web, and reason its way to conclusions that form the basis of a profile.
Here’s a simple example of how this might work:
If we repeat this process for millions of customers, we can create millions of profiles and customer profiles for use across the business.
With the help of smart machines and AI, Telcos can use their existing customer data or Network data to manufacture human-like profiles at scale. But these need to be done in a privacy-safe way to drive customer insight and scale personalisation. And when customers are treated like individuals, they are much more likely to engage.