Friday, January 12, 2018

The Light Bulbs Have Ears: Why Listening Is Voice-Activated Devices' Most Important Skill

If one picture’s worth a thousand words, why is everyone rushing to replace graphical interfaces with voice-activated systems?

The question has an answer, which we’ll get to below. But even though the phrasing is a bit silly, it truly is worth asking. Anyone who’s ever tried to give written driving directions and quickly switched to drawing a map knows how hard it is to accurately describe any process in words. That’s why research like this study from Invoca shows consumers only want to engage with chatbots on simple tasks and quickly revert to speaking to a human for anything complicated. And it’s why human customer support agents are increasingly equipped with screen sharing tools that let them see what their customer is seeing instead of just talking about it.

Or to put it another way: imagine a voice-activated car that uses spoken commands to replace the steering wheel, gear shifter, gas and brake pedals. It’s a strong candidate for Worst Idea Ever. Speaking the required movements is much harder than making them movements directly.

By contrast, the idea of a self-driving car is hugely appealing. That car would also be voice-activated, in the sense that you get in and tell it where to go. The difference between the two scenarios isn’t vocal instructions or even the ability of the system to engage in a human-like conversation. Some people might like their car to engage in witty banter, but those with human friends would probably rather talk with them by phone or spend their ride quietly. A brisk “yes, ma'am” and confirmation that the car understood the instructions correctly should usually suffice.

What makes the self-driving car appealing isn’t that it can listen or speak, but that it can act autonomously. And what makes that autonomy possible is situational awareness – the car's ability to understand its surrounding environment, including its occupant’s intentions, and to respond appropriately.

The same is ultimately true of other voice-activated devices. If Alexa and her cousins could only do exactly what we told them, they’d be useful in limited situations – say, to turn on the kitchen lights when your hands are full of groceries. But their exciting potential is to do much more complicated things on their own, like ordering those groceries in the first place (and, eventually, coordinating with other devices to receive the grocery delivery, put the groceries in the right cabinets, prepare a delicious dinner, and clean the dishes).

This autonomy only happens if the devices really understand what we want and how to make it happen. Some of that understanding comes from artificial intelligence but the real limit is the what data the AI has available to process. So I’d argue that the most important skill of the voice-activated devices is really listening.  That’s how they collect the data they need to act appropriately. And the larger vision is for all these devices to pool the information they gather, allowing each device to do a better job by itself and in cooperation with the others.

Whether you want to live in a world where the walls, cars, refrigerators, thermostats, doorknobs, and light bulbs all have ears is debatable. But that’s where we’re headed, barring some improbable-but-not-impossible Black Swan event that changes everything. (Like, say, a devastating security flaw in nearly every microprocessor on the planet that goes undetected for years…wait, that just happened.)

Still, in the context of this blog, what really matters is how it all affects marketers. From that perspective, voice interfaces are highly problematic because they make advertising much harder: instead of passively lurking in the corners of a computer screen, appearing alongside search results,  larded into social media feeds, or popping up unbidden during TV shows, voice ads are either front-and-center or nowhere. Chances are consumers will be highly selective about which ads they agree to hear, so marketers will need to gain their permission through incentives such as discounts and coupons. Gaining the consent required by privacy regulations such as GDPR* will be good practice for this but it will soon seem like child’s play compared with what marketers need to do on voice devices. So one change is marketers will need a new set of skills around creating aural ads and convincing consumers to agree to listen to them.

A related skill will be making those ads effective. Remember that people are vastly better at processing visual images than words.  That’s why we have the 1000:1 word:picture cliché. That efficiency is why visual ads can be effective even if people don’t focus on them – they are still being registered on some level and people will pay closer attention to those that look interesting at a glance. Aural ads will transfer much less information per moment of attention and chances are most of that information will be forgotten more quickly. We’re in early days here and there’s much to learn. But if you can buy stock in a jingle-writing company, do it.

Another obvious change will be that the device vendors themselves have more control than ever over the messages their customers receive. This gatekeeper function is already at the center of the business models for Amazon, Facebook, Google, Apple and others (increasingly including non-net-neutral broadband operators). But as fewer channels become available to reach consumers and as the channels themselves deliver fewer messages per minute, the value of those messages will increase dramatically. Insofar as separately-controlled devices compete for consumer attention, the device vendors will have even more reason to deliver experiences that consumers find pleasant rather than annoying. Of course, as I’ve argued extensively elsewhere, “personal network effects” make it likely that most consumers will find themselves dealing primarily with a single vendor, so actual competition may be limited.**

The gatekeepers’ control over their customers’ experience means that marketers will increasingly need to sell to the gatekeepers to earn the opportunity to reach consumers. What’s different in a voice-driven world is the scarcity of contact opportunities, which means that gatekeepers don’t have enough inventory (e.g., ad impressions) to sell to all would-be buyers. This isn’t entirely new: even today, impressions for Web display, paid search, and paid social are auctioned to a considerable degree. But a huge reduction in inventory (and the impact of serving ads that lead consumers to opt out, assuming they really have that option) will make the gatekeepers much more selective and, no doubt, raise prices. The gatekeepers will also have more conflicts with potential advertisers as they sell more services of their own, adding yet another level of complexity and more opportunities for deal making.

Finally, let’s come back to the sensors themselves. Assuming that the gatekeepers are willing to share what they gather, marketers will finally be able to understand exactly how consumers are responding to their messages. It’s not just that they’ll be able to know exactly who saw which messages and what the subsequently purchased. The new systems will be collecting things like heart and respiration rates, creating the potential to measure immediate physical reaction to each advertisement.  It almost seems unnecessary to point out that listening devices will also capture conversations where consumers discuss specific products, not to mention their needs and intentions. The grand mysteries of marketing impact will suddenly be exposed with thoroughness, precision. and clarity. The change will be as revolutionary as X-rays, ultra sounds, and CAT scans becoming available to doctors. As with radiology in medicine, these new information streams will require new skills that form the basis of entirely new specialties.

In short, voice-activated devices will change the world in ways that have nothing to do with the interaction skills of chatbots or ease of placing orders on Alexa. Marketers' jobs will change radically, demanding new skills and creating new power relationships. Visual devices won’t really go away – people are too good at image processing to waste the opportunity. But presenting information to consumers will ultimately be less important than gathering information about them, something that will use all the sensors that devices can deploy.

Who knew the sentient housewares in Disney's Brave Little Toaster were really a product roadmap?

*General Data Protection Regulation.  Have we reached the stage yet where I no longer need to spell it out?

** The essence of personal network effects is the value of pooling data to create the most complete information about each customer. In many ways, situational awareness is another way of describing the same thing.

Tuesday, January 02, 2018

What's Next for Customer Data Platforms? New Report Offers Some Clues.

The Customer Data Platform Institute released its semi-annual Industry Update today. (Download it here).  It’s the third edition of this report, which means we now can look at trends over time. The two dozen vendors in the original report have grown about 25% when measured by employee counts in LinkedIn, which is certainly healthy although not the sort of hyper growth expected from an early stage industry. On the other hand, the report has added two dozen more vendors, which means the measured industry size has doubled. Total employee counts have doubled too. Since many of the new vendors were outside the U.S., LinkedIn probably misses a good portion of their employees, meaning actual growth was higher still.

The tricky thing about this report is that the added vendors aren’t necessarily new companies. Only half were founded in 2014 or later, which might mean they’ve just launched their products after several years of development. The rest are older. Some of these have always been CDPs but just recently came to our attention. This is especially true of companies from outside the U.S. But most of the older firms started as something else and reinvented themselves as CDPs, either through product enhancements or simply by adopting the CDP label.

Ultimately it’s up to the report author (that would be me) to decide which firms qualify for inclusion.   I’ve done my best to list only products that actually meet the CDP definition.*   But I do  give the benefit of the doubt to companies that adopted the label. After all, there’s some value in letting the market itself decide what’s included in the category.

What’s most striking about the newly-listed firms is they are much more weighted towards customer engagement systems than the original set of vendors. Of the original two dozen vendors, eleven focused primarily on building the CDP database, while another six combined database building with analytics such as attribution or segmentation. Only the remaining seven offered customer engagement functions such as personalization, message selection, or campaign management. That’s 29%.**

By contrast, 18 of the 28 added vendors offer customer engagement – that’s 64%. It’s a huge switch. The added firms aren’t noticeably younger than the original vendors, so this doesn’t mean there’s a new generation of engagement-oriented CDPs crowding out older, data-oriented systems. But it does mean that more engagement-oriented firms are identifying themselves as CDPs and adding CDP features as needed to support their positioning. So I think we can legitimately view this as validation that CDPs offer something that marketers recognize they need.

What we don’t know is whether engagement-oriented CDPs will ultimately come to dominate the industry. Certainly they occupy a growing share. But the data- and analysis-oriented firms still account for more than half of the listed vendors (52%) and even higher proportions of employees (57%), new funding (61%) and total funding (74%).  So it’s far from clear that the majority of marketers will pick a CDP that includes engagement functions.

So far, my general observation has been that engagement-oriented CDPs appeal more to mid-size firms while data and analysis oriented CDPs appeal most to large enterprises. I think the reason is that large enterprises already have good engagement systems or prefer to buy such systems separately. Smaller firms are more likely to want to replace their engagement systems at the same time they add a CDP and want to tie the CDP directly to profit-generating engagement functions. Smaller firms are also more sensitive to integration costs, although those should be fairly small when CDPs are concerned.

There’s nothing in the report to support or refute this view, since it doesn’t tell us anything about the numbers or sizes of CDP clients. But assuming it’s correct, we can expect engagement-oriented vendors to increase their share as more mid-size companies buy CDPs. We can also expect engagement-oriented systems to be more common outside the U.S., where companies are generally smaller. For what it’s worth, the report does confirm that’s already the case.

If the market does move towards engagement-oriented systems, will the current data and analytics CDPs add those features? That’s another unknown. There’s already been some movement: four of the original eleven data-only CDPs have added analytics features over the past year.  But it’s a much bigger jump to add customer engagement features, and sophisticated clients won’t accept a stripped-down engagement system. We might see some acquisitions if the large data and analytics vendors want to add those features quickly. But those firms must also be careful about competing with the engagement vendors they currently connect with. Nor are they necessarily eager to lose their differentiation from the big marketing clouds.  Nor is there much attraction to entering the most crowded segment of the market with a me-too product.

So most data and analytics vendors may well limit their themselves to their current scope and invest instead in improving their data and analytics functions. That will limit them to the upper end of the market but it's where they sell now and offers plenty of room for growth.  Certainly there’s a great deal of room for improved machine learning, attribution, scalability, speed, and automated data management. If I had to bet, I’d expect most data and analytics vendors to focus on those areas.

But I don’t have to bet and neither do you. So we’ll just wait to see what comes next. It will surely be interesting.

*CDP is defined as a marketer-controlled system that builds a persistent, unified customer database that is accessible by other systems.

**To further clarify, customer engagement systems select messages for individuals or segments.  Analytics systems may create segments but don't decide which messages go to which segment.  And execution systems, such as email engines, Web content management, or mobile app platforms, deliver the selected messages. 

Tuesday, December 19, 2017

Surprising Results in Customer Data Platform Survey

The Customer Data Platform Institute (CDPI) recently surveyed its members about their customer-facing systems and CDP deployment plans.  (Click here to download the full report.)  While CDP Institute members are obviously not typical marketers (being smarter, richer, and better looking), the answers still provide some intriguing insights into the marketplace.

Let’s start with the general state of customer facing systems. One-third reported they had many disconnected systems, just over one-third reported (37%) they had many systems connected to a central platform of some sort (9% unified customer database, 9% unified database and orchestration system, or 19% marketing automation or CRM platform), 6% said one system does almost everything, and the remaining 23% said they had some other configuration or didn’t know.

I’ve compared these results below with several other surveys that asked similar questions.

The one thing that immediately jumps out in this comparison is the CDPI Member survey showed a much lower percentage of replies for “one primary system”. Otherwise, the answers across all surveys are very roughly similar, showing about 30% to 50% of companies having many connected systems and many disconnected systems. This suggests more integration than I’d expect, but it depends on how much integration those “many connected” systems really represent.

The CDPI Member survey also asked about plans regarding Customer Data Platforms. Nineteen percent had a CDP already deployed, 18% had deployment in progress, and 26% planned to start deployment within the next 12 months. The remaining 38% either planned to start after 12 months (4%), had no plans to deploy (19%) or didn’t know (14%). I haven’t seen any other survey that asks this question but have no doubt that CDP deployment and plans are much higher for CDPI members than the rest of the industry average.

Where things get really interesting is when we explore how the same people answered these two questions. At first blush, you’d assume the 19% with a deployed CDP would be the same 18% who said they had many systems connected to a unified customer platform, either by itself or with an orchestration engine attached. Not so much. Here’s the actual cross tab of the results.

What you see (in the yellow cells) is just 42% of the people who said they had a deployed CDP also said they had many systems connected to a unified database. If we allow that a deployed CDP could be present in companies where one customer-facing system does almost everything or the customer facing systems are connected to marketing automation or CRM, then 74% of the deployed CDPs are covered.

I take the remaining 26% as a healthy reminder that just having a CDP doesn’t guarantee all your systems will connect to that CDP, either by feeding data into it or reading data from it. In fact, we know that many CDPs support analytics without being connected to delivery systems, so this really shouldn’t come as a surprise.

On a more encouraging note, as the green cells highlight, a good majority of in-process and planned CDP deployments are at companies with many disconnected systems or many systems connected to a marketing automation or CRM.  Those are the companies most in need of data unification. So it does appear that the CDP message is reaching its target audience and CDPs are being used as intended.

The survey also asked about company revenue, business type (B2B vs B2C), and region. Comparing those with current systems and CDP deployment also gave some interesting and unexpected results. But there's no point in repeating them here since you can download the full report and see for yourself.  Enjoy!

Wednesday, December 06, 2017

Here's a Game to Illustrate Strategic Planning

My wife is working on a Ph.D. in education and recently took a course on strategic planning for academic institutions. Her final project included creating a game to help illustrate the course lessons. What she came up with struck me as applicable to planning in all industries, so I thought I’d share it here.

The fundamental challenge she faced in designing the game was to communicate key concepts about strategic planning. The main message was that strategic planning is about choosing among different strategies to find the one that best matches available resources. That’s pretty abstract, so she made it concrete by presenting game players with a collection of alternative strategies, each on a card of its own. She then created a second set of cards that listed actions available to the players. Each action card showed which strategies the action supported and what resources it required. There were four resources: money, faculty, students, and administrative staff.  To keep things simple, she assumed that total resources were fixed, that each strategy contributed equally to the ultimate goal, and that each action contributed equally to whichever strategies it supported. 

In other words, the components of the game were:

- One goal. In the case of my wife’s game, the goal was to achieve a “top ten” ranking for a particular department within a university. (It was a good goal because it was easily understood and measured.)

- Four strategies. In my wife’s game, the options were to build up the department, cooperate with other departments at the university, cooperate with other universities, or promote the department to the media and general public.

- A dozen actions. Each action supported at least one strategy (scored with 1 or 0) and consumed some quantity of the four resources (scored from 0 to 3 for each resource). Actions were things like “run a conference”, “set up a cross-disciplinary course” and “finance faculty research”.

- Four resources, each assigned an available quantity (i.e., budget).

As you can tell from the description, the action cards are the central feature of the game.  Here's a concrete example, where each row represents one action card:

The fundamental game mechanism was to pick a set of actions.  These were scored by counting how how many supported each strategy and how many resources they consumed.  The resource totals couldn't exceed the available quantities for each resource.  The table below shows scoring for a set of three actions.

 In this particular example, all three actions support "cooperate with other departments", while two support "build department" and one each supports "cooperate with other universities" and "promote to public".  Resource needs were money=8, faculty=6, student=5 and administration= 1.  Someone with these cards could choose "cooperate with other departments" as the best strategy -- if the resources permitted.  But if they were limited to 7 points for each resource, they might switch the "fund scholarship" card for the "extracurricular enrichment" card, which uses less money even though it consumes more of the other resources.  That works because, with a budget of 7 for each resource, the player can afford to increase spending in the other categories.

As this example suggests, the goal of the game is to get players to think about the relations among strategies, actions and resources, and in particular how to choose actions that fit with strategies and resources.

Although the basic scoring approach is built into the game, there are many ways my wife could have played it:

- Predefine available resources and let different players draw different action cards.  They would then decide which strategy best fit the available cards and resources. 

- Give different strategy cards to different players and put all action cards face up on the table.  Players then each choose one action card in turn, trying to assemble the best set of actions for their assigned strategy.

- Randomly select the resource levels at the start of the game and let all players use all action cards.  The winner is whoever first finds combination of actions that yields the most points for any strategy without exceeding the resources available.

- Split the class into two teams, gave each team two strategy cards and a set of action cards, and let the winner be whichever team finds the combination of actions that comes closest to using all available resources. (That’s the one she chose.)

Other rules are possible, along with refinements such as making some strategies more valuable than others at reaching the goal and making some actions more effective than others at supporting a given strategy. But my wife had ten minutes to explain, play, score and discuss the game, so the simplifications made sense for her situation.

What I like about this game is that it clearly identifies the elements of the strategic planning process and shows how they’re related. Specifically, it highlights that:

- different strategies can reach the same goal. Identifying available strategies and choosing among them is an important part of strategic planning that's often not clearly recognized.

- different actions can support different strategies. This has two implications: strategies are initially chosen in part based on what actions are available and, later, actions are evaluated based on how well they support the chosen strategy.

- different actions can compete for the same resources. In the short run, the combination of actions must be chosen to maximize the value achieved from the resources available. In the long run, resources are not fixed, so organizations can decide which resources they need to support the actions they need for strategic success.

- different strategies are best suited to different combinations of resources. This is the ultimate message of the game. Actions are just intermediaries to help understand how specific strategies and specific resources are related.

I hope you find this interesting and perhaps even useful. It’s more thought experiment than actual game.  But if you’re inspired to create your own physical version, do send me pictures.

Friday, December 01, 2017

2017 Retrospective: Things I Didn't Predict

It’s the time of year when people make predictions. It’s not my favorite exercise: the best prediction is always that things will continue as they are, but what’s really interesting is change – and significant change is inherently unpredictable. (See Nassim Nicholas Taleb's The Black Swan  and Philip Tetlocks' Superforecasting on those topics.)

So I think instead I’ll take a look at surprising changes that already happened. I’ve covered many of these in the daily newsletter of the Customer Data Platform Institute (click here to subscribe for free). In no particular order, things I didn’t quite expect this year include:

- Pushback against the walled garden vendors (Facebook, Google, Amazon, Apple, etc.) Those firms continue to dominate life online, and advertising and ecommerce in particular. (Did you know that Amazon accounted for more than half of all Black Friday sales last week?) But the usual whining about their power from competitors and ad buyers has recently been joined by increasing concerns among the public, media, and government. What’s most surprising is it took so long for the government to recognize the power those companies have accrued and the very real threat they pose to governmental authority. (See Martin Gurri’s The Revolt of the Public for by far the best explanation I’ve seen of how the Internet affects politics.)  On the other hand, the concentrated power of the Web giants means they could easily converted into agents of control if the government took over.  Don’t think this hasn’t occurred to certain (perhaps most) people in Washington.  Perhaps that’s why they’re not interested in breaking them up.  Consistent with this thought: the FCC plan to end Net Neutrality will give much more power to cable companies, which as highly regulated utilities have a long history of working closely with government authorities. It’s pitifully easy to imagine the cable companies as enthusiastic censors of unapproved messages.

- Growth in alternative personal data sources. Daily press announcements include a constant stream of news from companies that have found some new way to accumulate data about where people are going, who they meet, what they’re buying, what they plan to buy, what content they’re consuming, and pretty much everything else. Location data is especially common, derived from mobile apps that most people surely don’t realize are tracking them. But I’ve seen other creative approaches such as scanning purchase receipts (in return for a small financial reward, of course) and even using satellite photos to track store foot traffic. In-store technology such as beacons and wifi track behaviors even more precisely, and I’ve seen some fascinating (and frightening) claims about visual technologies that capture peoples’ emotions as well as identities. Combine those technologies with ubiquitous high resolution cameras, both mounted on walls and built into mobile devices, and the potential to know exactly who does and thinks what is all too real. Cross-device matching and cross-channel identity matching (a.k.a. “onboarding”) are part of this too.

- Growth in voice interfaces. Voice interfaces don't have the grand social implications of the preceding items but it’s still worth noting that voice-activated devices (Amazon Alexa and friends) and interfaces (Siri, Cortana, etc.) have grown more quickly than I anticipated. The change does add new challenges for marketers who were already having a hard time figuring out where to put ads on a mobile phone screen.  With voice, they have no screen at all.  Having your phone read ads to you, or perhaps worse sing a catchy jingle, will be pretty annoying. To take a more positive view: voice interfaces will force innovation in how marketers sell and put a premium on agent-based services that make more decisions for consumers. Of course, that's only positive if the agents actually work in consumers’ interest. If the agents also serve other masters – such as companies that pay them to send business their way – consumers can easily be harmed. But at least they’ll have more time for things other than shopping.

- Retailers focus on convenience. Speaking of shopping: retailers with physical stores continue to panic about the growth of Amazon and other ecommerce vendors. What I find surprising isn’t the panic, but that their main reaction has been to introduce innovations like “BOPIS” (buy online, pick up in store) that focus on shopper convenience. Nothing will ever be more convenient than ordering remotely and having stuff delivered, so this is a game they’re guaranteed to lose. It’s clear to me that the future of in-store retail depends on creating entertaining, enjoyable experiences, and specifically on human interactions that online merchants can’t duplicate. Those interactions could be with store personnel, friends, and other shoppers. I’ll violate my rule against predictions and say here that categories of in-store retail that aren’t inherently entertaining will vanish (think: grocery shopping except maybe for fresh produce).  But I'll hedge my bet by not saying when.

- Marketing clouds increase their share. I don’t recall seeing any actual data on this, but my distinct impression from talking with buyers is that the major marketing clouds (Adobe, Salesforce, Oracle) are being bought by more companies. This shouldn’t have surprised me: I’ve spoken for years about “Raab’s Law”, which says that integrated suites always beat best-of-breed components in the long run. It seemed briefly that marketing technology would be different, largely because cloud-based systems make integration so much easier than before.  The vast profusion of specialized martech systems seemed to support this view. But it’s clear we’re now seeing “martech fatigue” set in, as marketers tire of purchasing an endless array of new systems they then barely use. One bolt of lightning to illuminate this was a recent Gartner survey that found martech spend is now falling. This bodes ill for independent martech vendors and suggests that the long-awaited consolidation may finally be at hand. The question really is how long the momentum of wilder times will carry many of today’s martech point solutions before they fall.

- Interest in self-service technology. I’ve seen several recent announcements related to the idea that marketers and non-technical users in other departments will develop their own systems using varying types of advanced technology. Chatbots, predictive models, and entire business process integrations have all been offered as things business users could create for themselves, often with a little help from artificially intelligent friends. Bosh, I say. Marketers are already overwhelmed by the complexity of their tools and in particular by the challenges of connecting separate systems. The cloud and AI might make this easier but they don’t make it easy. The growth in marketing clouds shows marketers voting with their budgets to avoid integration. To be clear, what surprises me is that people think self-service will work or even that it’s desirable. They should know better.

- No Customer Data Platform acquisitions. Okay, maybe I’m the only person who thinks about this. But it’s still pretty odd that none of the big marketing clouds has yet purchased a CDP vendor. (The only deal I recall was Campaign Monitor buying Tagga and those are not major players.) I can think of many ways to explain this: cloud vendors don’t see the problem, they think they’re already solved it, they don’t want to admit they haven’t solved it, they don’t want to reengineer existing products to use a CDP, they prefer to buy companies with large market share, they think they can build their own CDPs, etc. But, ultimately, the big marketing clouds haven’t purchased a CDP because their clients haven’t pushed them for one. The marketing clouds will act quickly once they start losing deals because clients want CDP functions the vendors can’t provide. Maybe the limited degree of integration within the existing marketing cloud architectures is enough, or maybe buyers don't realize it's inadequate until after they’ve made the purchase.  I fully expect acquisitions to happen – oops, another prediction – but not very soon. And if the clouds continue to increase their share without adding a CDP, maybe it won’t happen at all.

- No uptake on the idea of “personal network effects”. There’s no question that I’m the only person thinking about this one. But I continue to believe the concept (described here) is central to understanding how things really work in today’s online economy. I'm surprised other people haven't picked up on it, especially as they pay more attention to the power of the walled garden vendors. For example, anti-trust regulators are struggling with the fact that firms like Google, Facebook and Amazon are not monopolies in the conventional sense, are missing the point that such firms can monopolize the attention (and, thus, purchases) of individual consumers. If anybody wants to co-author a Harvard Business Review article on this, let me know. (I’m serious.)

So much for surprises. Lest you get the impression that I’m always wrong, I'll list some things that haven’t surprised me one bit.

- No pressure on privacy. While many people keep expecting consumers to really start caring about personal privacy, I’ve never seen any reason to think that will happen. Experience has shown that even the smallest reward is enough for people to expose pretty much everything there is to know about themselves. Sometimes it doesn’t even require paying money; just a bit more convenience or recognition is enough. If anything, people are getting more used to everything being public and thus less concerned about keeping anything private. They suspect, probably correctly, that it’s a losing battle.

- Lack of unified customer data. Marketers have talked for years about the need for a complete customer view and the integrated, omni-channel customer experience that’s needed to support it. It’s possible there’s actually been some progress: while surveys used to show 10-15% of marketers said they had a unified view, I’m now often seeing figures in the 25-40% range. I don’t believe the real numbers are anywhere that high but maybe they indicate a little improvement. Even so, given the important assigned to the topic, you might expect the problem would mostly be solved by now. I’m not surprised it hasn’t been: building a unified view is tough, more because of organizational obstacles than a lack of technology. So things will continue to move slowly.

- AI bubble remains unburst. Surely it’s time for people to stop getting excited every time they hear the term “artificial intelligence”? Apparently not; pretty much every new product I see announced, including vacuum cleaners and doorknobs, has an AI component. People will eventually expect AI to be built into everything, just as they expect electricity, plastics, and other previous miracle technologies. But it takes a long time for people to recognize what’s possible, so I’m not surprised they still find even basic AI features to be amazing.

Wednesday, November 22, 2017

Do Customer Data Platforms Need Identity Matching? The Answer May Surprise You.

I spend a lot of time with vendors trying to decide whether they are, or should be, a Customer Data Platform. I also spend a lot of time with marketers trying to decide which CDPs might be right for them. One topic that’s common in both discussions is whether a CDP needs to include identity resolution – that is, the ability to decide which identifiers (name/address, phone number, email, cookie ID, etc.) belong to the same person.

It seems like an odd question. After all, the core purpose of a CDP is to build a unified customer database, which requires connecting those identifiers so data about each customer can be brought together. So surely identity resolution is required.

Turns out, not so much. There are actually several reasons.

- Some marketers don’t need it. Companies that deal only in a single channel often have just one identifier per customer.  For example, Web-only companies might use just a cookie ID.  True, channel-specific identifiers sometimes change (e.g., cookies get deleted).  But there may be no practical way to link old and new identifiers when that happens, or marketers may simply not care.  A more common situation is companies have already built an identity resolution process, often because they’re dealing with customers who identify themselves by logging in or who transact through accounts. Financial institutions, for example, often know exactly who they’re dealing with because all transactions are associated with an account that’s linked to a customer's master record (or perhaps not linked because the customer prefers it that way). Even when identity resolution is complicated,  mature companies often (well, sometimes) have mature processes to apply a customer ID to all data before it reaches the CDP. In any of these cases, the CDP can use the ID it’s given and not need an identity resolution process of its own.
- Some marketers can only use it if it’s perfect. Again, think of a financial institution: it can’t afford to guess who’s trying to take money out of an account, so it requires the customer to identify herself before making a transaction. In many other circumstances, absolute certainty isn’t required but a false association could be embarrassing or annoying enough that the company isn’t willing to risk it. In those cases, all that’s needed is an ability to “stitch” together identifiers based on definite connections. That might mean two devices are linked because they both sent emails using the same email address, or an email and phone number linked because someone entered them both into a registration form. Almost every CDP has this sort of “deterministic” linking capability, which is so straightforward that it barely counts as identity resolution in the broader sense.

- Specialized software already exists. The main type of matching that CDPs do internally – beyond simple stitching – is “fuzzy” matching.  This applies rules to decide when two similar-looking records really refer to the same person. It's most commonly applied to names and postal addresses, which are often captured inconsistently from one source to the next. It might sometimes be applied to other types of data, such as different forms of an email address (e.g. and The technology for this sort of matching gets very complicated very quickly, and it’s something that specialized vendors offer either for purchase or as a service. So CDP vendors can quite reasonably argue they needn’t build this for themselves but should simply integrate an external product.

- Much identity resolution requires external data. This is the heart of the matter.  Most of the really interesting identity resolution today involves linking different devices or linking across channels when there’s no known connection. This sort of “probabilistic” linking is generally done by vendors who capture huge amounts of behavioral data by tracking visitors to popular Web sites or users of popular mobile applications, or by gathering deterministic links from many different sources. They then build giant databases (or "graphs" if you want to sound trendy) with these connections.  Even matching of offline names and addresses usually requires external data, both to standardize the inputs (to make fuzzy matching more accurate) and to incorporate information such as address and name changes that cannot be known by inspecting the data itself.  In all these situations, marketers need to use the external vendors’ data to find connections that don’t exist within the marketers’ own, much more limited information. If the external vendor provides matching functions in addition to the data, the CDP is relieved of the need to do the matching internally.

In short, there’s a surprisingly strong case that identity resolution isn’t a required feature in a CDP.  All the CDP really needs is basic stitching and connections to external services for more advanced approaches.  As cross-device and cross-channel matching become more important, CDPs will be more reliant on external vendors no matter what capabilities they’ve built for themselves. One important qualifier is the CDP implementation team still needs expertise in matching, so they can help clients set it up properly. But while it’s great to find a CDP vendor with its own matching technology, lack of that technology shouldn’t exclude a vendor from being considered a CDP.

Thursday, November 09, 2017

No, Users Shouldn't Write Their Own Software

Salesforce this week announced “myEinstein” self-service artificial intelligence features to let non-technical users build predictive models and chatbots. My immediate reaction was that's a bad idea: top-of-the-head objections include duplicated effort, wasted time, and the potential for really bad results. I'm sure I could find other concerns if I thought about it, but today’s world brings a constant stream of new things to worry about, so I didn’t bother. But then today’s news described an “Everyone Can Code” initiative from Apple, which raised essentially the same issue in even clearer terms: should people create their own software?

I thought this idea had died a well-deserved death decades ago. There was a brief period when people thought that “computer literacy” would join reading, writing, and arithmetic as basic skills required for modern life. But soon they realized that you can run a computer using software someone else wrote!* That made the idea of everyone writing their own programs seem obviously foolish – specifically because of duplicated effort, wasted time, and the potential for really bad results. It took IT departments much longer to come around the notion of buying packaged software instead of writing their own but even that battle has now mostly been won. Today, smart IT groups only create systems to do things that are unique to their business and provide significant competitive advantage.

But the idea of non-technical workers creating their own systems isn't just about packaged vs. self-written software. It generally arises from a perception that corporate systems don’t meet workers’ needs: either because the corporate systems are inadequate or because corporate IT is hard to work with and has other priorities. Faced with such obstacles to getting their jobs done, the more motivated and technically adept users will create their own systems, often working with tools like spreadsheets that aren’t really appropriate but have the unbeatable advantage of being available.

Such user-built systems frequently grow to support work groups or even departments, especially at smaller companies. They’re much disliked by corporate IT, sometimes for turf protection but mostly because they pose very real dangers to security, compliance, reliability, and business continuity. Personal development on a platform like myEinstein poses many of the same risks, although the data within Salesforce is probably more secure than data held on someone’s personal computer or mobile phone.

Oddly enough, marketing departments have been a little less prone to this sort of guerilla IT development than some other groups. The main reason is probably that modern marketing revolves around customer data and customer-facing systems, which are still managed by a corporate resource (not necessarily IT: could be Web development, marketing ops, or an outside vendor). In addition, the easy availability of Software as a Service packages has meant that even rogue marketers are using software built by professionals. (Although once you get beyond customer data to things like planning and budgeting, it’s spreadsheets all the way.)

This is what makes the notion of systems like myEinstein so dangerous (and I don’t mean to pick on Salesforce in particular; I’m sure other vendors have similar ideas in development). Because those systems are directly tied into corporate databases, they remove the firewall that (mostly) separated customer data and processes from end-user developers. This opens up all sorts of opportunities for well-intentioned workers to cause damage.

But let’s assume there are enough guardrails in place to avoid the obvious security and customer treatment risks. Personal systems have a more fundamental problem: they’re personal. That means they can only manage processes that are within the developer’s personal control. But customer experiences span multiple users, departments, and systems. This means they must be built cooperatively and deployed across the enterprise. The IT department doesn't have to be in charge but some corporate governance is needed. It also means there’s significant complexity to manage, which requires some sort of trained professionals need to oversee the process. The challenges and risks of building complex systems are simply too great to let individual users create them on their own.

None of this should be interpreted to suggest that AI has no place in marketing technology. AI can definitely help marketers manage greater complexity, for example by creating more detailed segmentations and running more optimization tests than humans can manage by themselves. AI can also help technology professionals by taking over tasks that require much skill but limited creativity: for example, see Qubole, which creates an “autonomous data platform" that is “context-aware, self-managing, and self-learning”. I still have little doubt that AI will eventually manage end-to-end customer experiences with little direct human input (although still under human supervision and, one hopes, with an occasional injection of human insight). Indeed, recent discussions of AI systems that create other AI systems suggest autonomous marketing systems might be closer than it seems.

Of course, self-improving AI is the stuff of nightmares for people like Nick Bostrom, who suspect it poses an existential threat to humanity. He may well be right but it’s still probably inevitable that marketers will unleash autonomous marketing systems as soon as they’re able. At that point, we can expect the AI to quickly lock out any personally developed myEinstein-type systems because they won’t properly coordinate with the AI’s grand scheme. So perhaps that problem will solve itself.

Looking still further ahead, if the computers really take over most of our work, people might take up programming purely as an amusement. The AIs would presumably tolerate this but carefully isolate the human-written programs from systems that do real work, neatly reversing the “AI in a box” isolation that Bostrom and others suggest as a way to keep the AIs from harming us. It doesn’t get much more ironic than that: everyone writing programs that computers ignore completely. Maybe that’s the future Apple’s “Everyone Can Code” is really leading up to.

*Little did we know.  It turned out that far from requiring a new skill, computers reduced the need for reading, writing, and math.