Marketing Analytics Problems (Solved)
- The main reason companies give up on analytics is because they do not get the proper results out of all their hard work
- Sometimes analytics initiatives fail because they are defined wrong, because you didn’t ask the right business questions first.
- Asking the right questions also helps to avoid death by KPI. Too many KPI’s cause businesses to be indecisive on what to measure and what actually impacts their business
- There needs to be a culture shift, from the top down, to commit to analytics and think of the future not the past when making decissions and investments
What follows is a lightly edited transcript of Episode 18 of the Inevitable Success Podcast.
Transcript:
Damian Alright, so today we thought we would touch a topic that is very near and dear to both of our hearts, and it is, “Why is it that some businesses struggle to commit to using analytics in their marketing?”
Stephen There are many reasons. Um, the number one reason, that comes to my mind, is that well, one is that they don’t get the proper result out of all these analytical endeavors, if you will. So they have this notion that, yea, this thing is complex…it’s expensive. It takes a long time, and you know, “I talk to analysts and they are talking about things that I don’t understand, so I just automatically hate it.”
Another thing is that a lot of successful companies, they think that they have been successful without analytics, so to introduce this new element, it’s bit of a cultural war. Like your gut feeling versus actual numbers. It’s a bit of a training on the non analytical people as well, meaning, you cannot just hire a bunch of analysts and say “Hey, you’re an analytical company!” You have to have a culture of consuming such analytics, without that, it fails. There’s no question about that.
Damian So, you know, when we say the word, you know, “use analytics,” what does that actually end up meaning?
Stephen You know, I think of it in one way, you may think of it another way. I think we’re probably thinking about it more similarly. When I think of it, it is, you know, informed decision making, but maybe there’s a few different angles that might mean different things to different people.
I look at it from the timeline point of view. And that scares people to say, “Oh we’re talking about predictive analytics.” Yes I am, but it’s not the scary thing. Most interesting notion is what they call prescriptive analytics.
Damian That’s interesting. What is that anyway?
Stephen It’s totally bogus. It’s just that you know some people, who are probably some stat geeks, made up this chart, and it’s a pyramid chart. At the bottom, they have business analytics and reporting. Ok, fine, it seems relatively simple, but is it that simple? I don’t think so. Number two is a descriptive analytics.
Damian It’s like a food pyramid for…?
Stephen Yea, it’s kind of like a pyramid….Exactly. At the bottom you have a BI and…
Damian Got it. So we have an analytics pyramid scheme.
Stephen Second tier, is what they call descriptive analytics. That’s where segmentation, profiling, clustering…those wonderful things fit in. In other words, I know what’s going on. The BI reporting will tell you… that yeah, “What is the best time to blast e-mail? Thursday morning is the best time.” Or “What are the things that are happening right now? What is going on right now?” So I blast this e-mail. There’s a clickstream open…All those wonderful things. That’s BI. Now descriptive analytics is, “Who’s behind all these actions? What do they look like? How many groups of people, how many pockets of people, am I dealing with here?”
For example, we just consulted another company yesterday, and they have a very dichotomous product line. Their flagship product is like $1800… if you look at their customer value, the average customer value is like three hundred dollars or less. That means you have very dichotomous pockets of people in it, where some people may buy all those expensive things and don’t ever come back. The second tier of people may be coming a lot, but their spending level is less than three hundred dollars a transaction. So knowing who’s behind all this, is what we call descriptive analytics. So far so good right?
Then the future questions come in, now, that gets really difficult. Like, “Who is more likely to do what? What product will be more popular? How many units of this X Y Z product am I going to sell, so that I can manage my supply chain properly? Allocate the budget properly?” And all those wonderful things that come with a budget. The simplest kind, and in marketing the way we normally talk about is, “Most likely to do something.” It could be great product affinity, it could be the channel responsiveness. We can even predict things like who’s going to be leaving our customer base. This could be called a “trend prediction,” if you will. Or “I signed up somebody, and I want to know his lifetime value before he spends another dime.” Well it’s very difficult. But you could do that. That’s called the lifetime value model. Everything that I just said here, including forecasting model, is about prediction of the future. Just like your weather report by the way. And people think the analytics is difficult. No, no…You are a consumer of predictive analytics every day.
If you woke up in the morning and say “OK so I’m going to New York City today, and when I listen to the radio or check out my iPhone, to see what is the weather like today? And yesterday they predicted 70 percent chance of shower in the afternoon, right? I don’t have to know what they had to do, to do the actual analytics. As a consumer that information, all I care is that with a 70 percent chance of showers, I’m definitely taking my umbrella.
Damian Right.
Stephen I want people to think about it that way. I don’t want them to think about how difficult it is to build a model or whatever. And also with living in an age of machine learning, all this fast modeling, in database, and all the terms that we say all the time. It means it got easier easier, and simpler, over time. To the consumers of this information, this should just understand that “oh there’s a thing called predictive analyst which predicts the future. And you know what? I can be a consumer of it, because I know you have a thousand data points, but what I really want to know is who is more likely to respond to this offer.”
Damian Right. And it’s not. I mean, I think we use the word “predict” loosely. I think it’s probabilities.
Stephen Probabilities. That’s another way to put it.
Damian Yea, you know, when you made that analogy about 70 percent chance of rain, that’s prediction, that’s the projected predicting. But it’s such a probability. In the database, you may have somebody that, you know, there’s a 70 percent chance that they would make a purchase in this category. You know I’m trying to think of what’s the analog to bring my umbrella. You know.
Stephen Well for a marketer the questions are, by the way questions are always asked by the customers or the users of information. And by the way that’s the first thing. In fact, let’s talk about that first. Why the analytics fail is because they’re not asking the right question.
Damian Yeah.
Stephen You cannot just throw a whole bunch of data to analyze and you know “just find the way to make more money for us.” Now they’re not wrong, because why do any analytics? Two reasons. That’s it. Either increase their revenue for marketing or decrease cost..therefore increase profit.
Damian We could end the episode now. That’s it. We cracked it.
Stephen But there are many ways to skin this whole thing, right? And so what can I say, asking the right question, is the most important thing. And by the way in our company, that’s what we do! First reaction that we get sometimes, and I’m not saying all of them are like that, when you show a lot of reports out of all this data cleansing that we do, in decent shape of reports and charts and all that…If you show that to a data geek, they get they get excited but a lot of people get scared, like, “Oh I don’t know what this means.” And what that means is that “Ok, I don’t understand what’s going on, or how to read this thing,” into “I don’t know what to do about it.” In a way, the job of an analyst is to do both. “This is what it means. There are all these reports of what it means.”
Meaning that you know what, “You have a seasonal problem” or “You have a one time buyer problem,” or “You have a value problem” or “You have an acquisition cost problem, because you’re not getting enough response rate out of all the effort that you’re doing.” Or “Maybe you have a channel problem… maybe you don’t have a channel problem for e-mail, but you do have a problem for direct marketing for such things.” These are the findings that analysts have to find out. Again this would be the be the BI reporting and the combination of descriptive analytics.
Then I kind of knock down the whole notion of prescriptive analytics. Prescriptive analytics is nothing but…“Okay, these are the issues that you’re dealing with, so what are you going to do about it?” As a prescription, as a result of a prescription, you may say, “Oh you need a model,” Which is a prediction right? Or “You don’t need a model. You don’t have a much bigger thing, but you really have to differentiate different groups of people so let’s do more segmentation before you do anything.” Everything that I just said here’s a form of prescription, in a way. So if you don’t do it this way, in other words, find the business goal, define the problem statement. This is the problem that we want to solve. You have to do that first. Only then you have to prescribe what you’re going to do. Otherwise what happens now is that “We have a lot of data, so we’ll just build models and we’ll just do these things.”
In fact, I was interviewing some Ph.D. level statistician and I just rejected him at the phone interview. I didn’t meet the guy, because whenever I meet with mathematically trained people, I go reverse. That can he understand what this is all about?
So I said “Ok you build a lot of models. Tell me, why did you build these models?” And he couldn’t answer the question. His answer was more like “Just because.” Well, that’s not good. And he kept resorting back to how he built it. What kind of methodology he used, or what kind of a hardship that he went through to build this database and all that. I don’t care. Just like I do not care why or how the weather prediction happened this morning. I just don’t care. You have to understand the business reason behind all these things. That means, you must start by asking the right questions. And this is not just the job of an analyst, but business owners as well. So first question. Ok, well we have a lot of issues here, but in a general sense, I know you want to make more money. I get that…we all do. If you have a retention issue, and customer retention versus customer acquisition…which problem do you want to solve first? At least you have to understand that. In other words, it’s like going to your doctor’s office.
Damian That’s a good question.
Stephen You cannot say that “I don’t feel well.” You know what, some doctor with a superpower may say that “Oh you know, you just need this.” I’ve seen doctors like that… but more like acupuncturists, than those guys. But normally you have to tell them what is wrong with you, and they will work together to find the solution. Whether you take drugs or get surgery…whatever it is. That process is really about the same, when it comes to analytics. [00:10:33] So when it fails, I say, “Did you even ask the right question? How do you define failure?”
Damian Right.
Stephen For example, there are people who say that “Oh, I have spent all this money to build this model, and all these club companies, built amazing targeting and all that. And I only got like 1 percent response rate.” Ok, well why do you call it a failure? Sounds like a decent, not great, but decent response rate…maybe you expected more. I don’t know.
Maybe it was not as good as last year or maybe it was not as good as another channel, or maybe it was not as good as another product category. I don’t know what it is. You cannot just say that it doesn’t feel right. Sometimes the goal is just greed. It’s like “Oh I didn’t break even.” Well then please tell me that upfront so that we do not design an expensive solution, because sometimes the assumption is that you know that you’re not going to break even the first time. But the goal is to get somebody in and cultivate that relationship. Or if you don’t want to do that, then let us know so that we can properly define the problem statement and find a solution that fits your business goal. If you don’t do that, even very successful analytics may look like failure, because you didn’t set the benchmark properly.
So I just wanted to throw that in there, because sometimes failure in itself is defined wrong, because you didn’t ask the business question first.
Damian So I think I’m gonna write this one down. [00:11:57] One of the reasons that businesses, you know, don’t commit to analytics, is because they’re just not asking the right questions.
Stephen It happens all the time. And then related to that is, sometimes it’s a top down decision. Maybe their CEO or CMO went to some conference and said “Yea, this is an age of machine learning and big data.” If you’re gonna do this, okay, maybe the top people committed to it. But did you cultivate the culture of analytics in your own company? That’s another question. I cannot name names because I had this meeting before I came here. This company is a really large telecommunication company. They have data collected from everywhere…call center, even the cable box, all the order history, you name it, everything. They say the terabytes of data flow through their system all the time. And yet, people still make decisions based on their gut feelings, or worse, they don’t even think too much. Whatever the marketing that they do, is a copy and modified version of last year’s plan. Now if everybody in your company in the marketing department starts copying and pasting, and just modifies a little bit every year, your marketing plan is fundamentally the same as the 1970s. Seriously speaking. And how do you wonder…you are going to live with a new media, new consumption…all these millennials behaving differently. How are you going to react to that? You cannot just batch and bless it, and mainly hope to God that somebody clicks. You get to have a culture of data consumption.
Now, long story short I didn’t have anything to sell, because the CMO and CTO there are doing all the right things, but I said, “Look, you don’t have the culture of analytics…that I cannot help. This is an internal issue here.” Maybe you want to have a different KPI. Maybe the whole thing should be measured more properly so that people get aware that repeating the same old things is not good. You know what that means? You know that the bottom pile, that I describe as a B.I. in reporting…maybe you should start there. Maybe that is wrong. Maybe you’re measuring all the wrong things.
Damian Exact measurement protocols.
Stephen Exactly…you don’t get into the right behavior, because you’re looking at the wrong KPI. So sometimes it’s as simple as that.
Damian Yea, on that too, I see a lot of organizations get stuck at that level too. They are measuring everything, but making decisions ordered on nothing, and that can be pretty expensive too.
Stephen Extremely expensive, and I saw this in large corporations as well. And I call that “death by KPI.” Too much is not good either, by the way.
Damian Right.
Stephen And why do you have too many KPIs? Because you don’t know what you want to measure.
Damian Right.
Stephen Why don’t you know that? Because you didn’t define the goal yet.
Damian If everything is important, nothing is.
Stephen Exactly the point. So death by KPI, is that everybody is looking at the excel spreadsheet, but nothing changes. Again, there’s a definition of KPI coming from the business goals, and once you buy into it, and just define like three or four major KPI for each individual or department…but you’ve got to have a culture of non blame, no blame, but fixing things. In fact, I’ve seen, and maybe this is a failure.
I’ve seen a major food distribution company in Korea, and had some friends there who are pioneering all this, like a big data usage for the company. And I don’t go to Korea all the time, although my family is from there. And I heard that they just got rid of that department. You know what happened? Basically these guys…this again, I’m talking about the culture of analytics…just listed the findings. This is what we see through the data. And this is what we have to fix. The owner, the CEO, and the CMO of the company started using that as an evidence to scold people…“Hey, how come you guys were doing it wrong for the past 10 years?” And you know what, that’s not a culture of analytics. That’s a blame game. I’m sorry. This is about the future. Of course they did not know what to do, because you know what? You did not hire any analysts, did ya? So how do people know that? You cannot use analytics for the web as a weapon to blame somebody. So it became a political battle.
So everybody who got blamed made sure that they got rid of the analytics department, and they won the battle…the department is gone. What a shame. But the point is, cultural war is very important here. You have to have a very, how do I put it, analytical mindset, even as a non-analyst, that you know what? This is all about logic. You cannot go back and change the past. This has to be a future-affording action, based on analytics, not about criticizing the past. That’s one. And you’ve got to be able to prescribe the right things and you know what, [00:16:33] in that case you’ve got to really bring some professionals here, because one of the things that you know as an analyst, that we deal with, is that people get scared. It’s like “Yea, don’t tell me all this – like an eight page stab.”
In fact if you look at the raw report, it could be like pages and pages, by timeline, by channel, by source, or whatever, right? Good analysts should be able to summarize the findings within a few pages, and also be able to talk about…so what are you going to do about it?
Damian You know, when I was first learning and getting into this space, I took a course by this guy named Avinash. A lot of people know him. He is Ockham’s Razor blog. Used to be Google’s evangelist for Google Analytics. And, you know, one of the things that he always used to say is that, you know, really good reporting, yea it has a chart and stuff, but it also has a summary…like there’s words on really good reports, from a good analyst. And I always, always remember that, because you know, we always think of like all reports is like dashboards and charts. But he said, you know, really great reports, is to have some words on them, and they tell you what their point of view is and what you should do. That’s interesting.
Stephen So I don’t judge….I’m a geek anyway, but I’d rather look at the charts. I mean the raw chart and some pie charts anyway, right. But I guess I’m a geek, but that aside, I wrote a column once. The title’s called, “Every number on your report should be good, bad, or ugly.” Now it sounds like a funny thing to say, but this is the hardest thing to do. If I say that you know what, your click-through rate went up by one point, one percent this year, year to year. Is that a good or bad thing?
Damian I have no idea.
Stephen Exactly. Your job as an analyst, is to know it. Now you have to have a lot of information about it. In other words…ok so what’s the baseline here? And how much did you spend on that channel? If you spent a lot of effort and did all these things, and it only grew like one point, one percent… Hey, that’s pretty bad actually. Or if you didn’t do anything, and did minimal things, and it grew that much, it’s like…Hey, I commend you! This is great, you had a growth…you didn’t lose.
That’s great. In a competitive market. Knowing the baseline, whatever that is, in fact when I don’t know what I’m talking about, and when I can even get that out of my client, I kill them with love. I do all of it. Let’s compare it quarter to quarter, month to month, week to week, year to year, and then have a baseline, multiple ways. Company as a whole to different channel, or different guys in the industry…the kind of numbers that I can quote somewhere. [00:19:32] Now that way you have a better idea that, “Mmmm, how well am I doing.”
Just like when you send your kids to school, and somebody just threw in a score and say that his score is an 88, “Ok, is that good? I don’t know what the maximum score is. Is that curved? Is it scale? I don’t know that.” So therefore you have to have a reference. So that means, analysts, if you want to be in the game and contribute to this culture of analytics, it’s to understand your business goal and have the right reference. What it matters…compared to your budget or compared to last year? Hey it depends! I’ve seen organizations where they look at all of the above. What happens now is that it becomes a blame game, again depending on who’s saying it, and sometimes they’re right or wrong, but is twisted so much that, well it could be good, right, depending on who’s saying it and that’s not the culture of analytics.
Damian One of the things that you said there, and I don’t know if this is what you mean when you say the good, bad, or ugly. It almost sounded to me like it’s got to hit you in the face.
Stephen It does.
Damian And anything else that doesn’t have that reaction…You’re probably wasting attention, of the people that you’re speaking to.
Stephen Yes. And it’s not going to lead to any change.
Damian Exactly. It’s like, “Let’s not talk about that.”
Stephen So good, bad, and ugly is, in other words…numbers are…by the way, numbers scare people. In fact, I read somewhere that people are very much afraid of death, but even more than that, they’re afraid of public speaking, and even more than that they’re afraid of numbers.
Damian I thought you were going to say something like, the number 7 terrifies people.
Stephen Can you imagine a number with all lot of decimal places right? What does that mean?? So the trick that we use as an analyst, is to put it on the scale. Anybody can understand a one to five scale, one to ten scale… just like 70% chance of showers. And by the way, I think the weather forecasters dumb it down for us too.
Damian I’m sure they do.
Stephen If the model says seventy-two point three. Do you really need to know that, to change your mind about bringing the umbrella or not? Right?
Damian Right.
Stephen Did you notice that on Netflix, by the way, that if you rent their DVD, their revalidation scale remains to be five scale – One star being “Oh I hated that movie,” Five star means “I love that movie,” right? Online, though, on-demand streaming service, they changed that to a thumbs up, thumbs down. Because it’s easier for both…the person who has to report that. The guys who watch the movie and say “Oh I thumbs up, thumbs down,” right? And the analysts too. Imagine some machine is building a model right? And maybe the machine decided, I don’t need to know five scale point. I’m going to convert it to yes and no anyway. So I’ll just collect a yes and no.
Damian And that takes some subjectivity, actually, out of the variables.
Stephen If you have a very dichotomous yes and no answer all over the place, now it’s a scale game. I’m going to evaluate thousands of movies that way, and then maybe the prediction result is the same because I do recommend this…or not to you or to me, right?
So, therefore, why is it possible? Because they know the goal. If you don’t know the goal, then your instincts say “Oh I’m going to want to take the most granular data and hoard it as much as possible.” That was by the way, very much of the essence of the big data movement a few years ago. What was the big data movement? And you know, I’m sure you heard about that the 3-D’s of the big data right? What is it? Velocity, oh she move fast, right? Variety, and all kinds of data, right? Movement data, purchase data, viewing data, all that, right? And then there’s volume. Oh it’s big data…a lot of data is moving through the system, that’s a good thing. I’ve never met a guy that’s made tons of money just doing it that way.
Because just because there are many records moving through the system does not make you money, and going back to that cable or the telecom company example. So why do you have terabytes of data moving through your system every second? If nobody is consuming, and if the analysts did not make that consumable, as simple as, “Do you recommend this to that guy? Or not?” It should be that simple. And it comes from defining the goal properly. Is this about recommendation or is it about predicting the next quarter revenue? We’ll have that discussion because those two questions require a very different set of data, where even if the similar sets of data flow through it, the way you look at the data is vastly different. So can you imagine the analyst not knowing, oh, what is this about?
I just want to keep clean data…fine. But can you imagine how much you would cost to clean every bit of data? The trick is, know the goal and clean the data, only as much as you need to. So much cheaper. Ten times cheaper.
Damian And probably only collect as much as you need to.
Stephen That’s true too. Data hoarders are the enemies. I use the hoarding intentionally by the way…hoarding is bad.
In fact, we had this discussion around, like the first dot.com boom, and we had the choice because my first thought was about collecting the transaction that we kind of knew that. But what people say that they do and what actually people do, are two different things. If you’re in a prediction business, not just “oh he’s in love with this product.” You know what a lot of women are in love with Italian handbags, that cost like three thousand dollars. Is she a buyer of that product? Now that’s a very different question, isn’t it?
Damian Yeah I love Ferrari’s.
Stephen No kidding!
Damian I don’t own one.
Stephen By the way, do you really need a Ferrari? Or do you want a Ferrari? Those are two different questions too.
Damian You know I think I need it! Sorry, I threw you for a loop there for a second.
Stephen Actually when I was younger, I thought that I needed it too, but anyways, that’s a different discussion right? But when you get into it, if you know what you’re supposed to do with the numbers, then you don’t need to collect too much data. So we stuck to the transaction better than we had, but it’s all about the sentiment, about the election result, and so forth, by the way, where the new transaction is involved. [00:25:37] Then, of course, you have to collect different kinds of data.
Damian I think you said it almost early, in the first few minutes, of this. What data do you collect? Well, you ask the right questions.
Stephen That’s right.
Damian Right.
Stephen You don’t need to hoard it.
Damian Then you’ll figure out what data you need to answer that question.
Damian Yeah. I think, you know, a lot of this is, like you said, the culture component. In our recording room, where our fourth core value in our company, is actually: “Be curious, have passion for learning, and synthesize.”
Stephen That’s right.
Damian I think that actually does a pretty decent job of summing up the type of culture you’d have to have, to make a commitment to analytics. You know, you have to have the curiosity to ask the right questions, and actually be a passionate person, the person that wants to get to the bottom to find out, “How do you answer that?” And then you actually have to do that last part too, which is to synthesize something. Take two disparate things and come up with a third thing that didn’t exist, that adds value. You know?
Stephen That’s a top-down culture change by the way. Culture changes are revolutionary; either for a country or government or companies…it’s revolutionary. That means you have to have the commitment from the top. Now, why do we need that? Because we have to force the change, and you have to hire a different kind of people to make that change. You cannot just do all this analytical transformation if you will. But the people who think that this is their problem, meaning that this number that comes out, maybe that will get me fired. Please don’t think that way.
Stephen Maybe, maybe our goal is to make you sell better. Maybe our goal, maybe oh, you’re in sales, and you don’t have an understanding of analytics, but you know what? Trust me when I say this, you have a hundred leads, but I’m going to rank order them from the most likely to buy, to the least likely to buy them. Who would you call first? You don’t have to look at a hundred variables. I’ll give you one square to sort by. Why wouldn’t you have it? I’m going to give you the Glen Gary, Glen Ross lead, in the form of scores. What is there not to like? Because more accountability follows it? No. So, therefore, the cultural change should really happen from the top. Get some commitment, and if you know the goal, believe me when I say that it will save a lot of money because you’re not going to hoard a lot of data in your database.
Damian Well put. Well, I’m sure on the next episode, we’re going to peel that down even more. As we always do.
Stephen Let’s do that. This was a really fun interaction because as a longtime analyst it was really bothering me why people don’t believe and don’t commit to this. So I thought a lot about this, so thank you for this line of questions.
Damian It’s great Stephen. Bye Bye.
