Speaker 2: So today we are looking at something that, ⁓ well, every single person listening to this has probably interacted with just this week. ⁓ absolutely. Right. We're talking about customer feedback surveys. We have this really comprehensive white paper from SupportLogic to go through today. Speaker 1: Absolutely, multiple times a week, I bet. Yeah, and it pulls together some heavy hitting academic literature too. Speaker 2: Exactly. They cite research from the University of Groningen, multiple analyses from the Harvard Business Review, plus a ton of real world industry data. Speaker 1: And the core argument here is just fascinating. Speaker 2: really is. Basically, they're saying the entire 1 to 10 scale that we've built the modern service economy on might be completely fundamentally broken. Speaker 1: challenges the very foundation of how organizations even measure success today. The source material does an excellent job synthesizing the history of these metrics, the statistical realities, and ⁓ the technological paradigm shift that is basically rendering the traditional survey obsolete. Speaker 2: And knowing how to interpret this data is crucial for you, whether you're running a massive tech enterprise or just managing a small team. Speaker 1: or even just as a consumer who is tired of taking them. Speaker 2: Yeah, I mean, even on a personal level, I actually noticed this exact problem recently when I decided to create a podcast. Yeah, when you create a podcast, you just become completely obsessed with feedback. You constantly ask your listeners to rate and review the show so you can gauge if the content is actually landing. Speaker 1: ⁓ really? Because you need that validation metric. Right. Speaker 2: but you quickly realize those ratings only tell a tiny fraction of the story. The people who actually leave reviews, they don't always represent your average listener. And the corporate world is facing that exact same dilemma, but on a multi-billion dollar scale. Okay, let's unpack this. Speaker 1: No, they definitely don't. Well, before we dissect the specific metrics, really have to understand the fundamental shift in the economy that, you know, caused this obsession with feedback in the first place. Speaker 2: Right, the transition to the subscription economy. Speaker 1: Exactly. The white paper highlights this massive transition specifically looking at software as a service or SAWS. Speaker 2: because the old model was pretty simple. You bought a CD-ROM in a cardboard box. You installed the software on your computer. That company already had your money. They didn't really have to interact with you again until they tried to sell you the next version three years later. Speaker 1: And that was it? Yeah. If we connect this to the bigger picture, earning loyalty is no longer a one-time transaction. In a subscription model, it's a month-after-month battle to keep your business. Precisely. And with the rise of open-source software and global competition, the software product itself has, in many cases, just become commoditized. The actual code or the core feature set isn't really the primary differentiator anymore. Speaker 2: because you can just cancel anytime. meaning the real value has shifted entirely to the support and services side of the business. Speaker 1: Yeah, customer support used to be viewed by executives merely as a cost center. Speaker 2: just an annoying expense that needed to be minimized. Speaker 1: Exactly. But now, support is the primary driver of customer retention. Think about it. If your customers can easily migrate all their data to a competitor service over a single weekend, your support experience is often the only thing keeping them loyal. Speaker 2: Which totally explains why companies suddenly became so desperate to measure how good their support actually was. Right. Speaker 1: which leads us to the alphabet soup of metrics. Speaker 2: Let's start with the first one, the white paper tackles. CSAT or customer satisfaction. It's classic. Yep. Used by nearly 80 % of customer service organizations. It's that standard five point scale, usually ranging from very unsatisfied to very satisfied. Speaker 1: one that always pops up right after you finish a chat with a support agent. Speaker 2: That's the one. And while it seems incredibly basic, the research cited in our sources reveals a really surprising statistical reality about CSAT. Speaker 1: It does, because you might assume an organization needs a massive multi-page survey probing every little detail of the interaction to extract any meaningful insight. No, but the data shows that a single overall satisfaction question actually has a 90 % correlation with those massive, complex surveys. Speaker 2: Nobody wants to those out. Wow, that is a massive relief for anyone who hates filling out 20 question forms. So that one simple question is doing all the heavy lifting for the data analyst. Speaker 1: It is, and the way those analysts use the results is even more telling. The industry relies heavily on a concept called Top-2-Box CSAT. Yeah, it means organizations often ignore the average score entirely. Instead, they just look at the fraction of customers responding with the two highest points on the scale, the fours and the fives. Speaker 2: Top Two Box. Just focusing on the most satisfied people. Speaker 1: Exactly. Statistically, focusing strictly on those most satisfied customers is highly predictive of retention. It provides a much clearer signal than trying to calculate some muddy average across the whole spectrum of respondents. Speaker 2: That methodology actually makes a lot of sense. mean, if someone gives a product a three out of five, they're probably just indifferent. ⁓ Right. But let's move to the absolute heavyweight champion of corporate boardrooms, NPS or the Net Promoter Score. Yes. This metric is everywhere. It was created by Frederick Reichel back in 2003, and it's that zero to 10 scale asking, how likely are you to recommend us to a friend or colleague? Speaker 1: Indifference doesn't drive loyalty. That's the big one. And the scoring methodology of NPS is what makes it so unique and simultaneously so incredibly controversial in the academic world. Speaker 2: Because you don't just average the numbers together, Speaker 1: at all. The system forces you to group the responses into three rigid clusters. If a customer scores you a nine or a 10, they are labeled a promoter. If they score a seven or an eight, they are a passive. And if they score anywhere from zero all the way up to six, they are categorized as the detractor. Speaker 2: And to calculate the final score, a company takes the percentage of promoters and subtracts the percentage of detractors. Yes. And the passes are completely thrown out of the equation. Wait, hold on. In what universe is an 8 out of 10 considered a non-factor? If a movie gets an 8 out of 10, it's a certified hit. Why does the corporate world just throw that data away? Speaker 1: entirely discarded. The underlying philosophy was that only overwhelming enthusiasm drives organic growth, and anything less than a nine is essentially just polite indifference. that rigid clustering leads to some truly bizarre counterintuitive math. Imagine a scenario where every single customer rates a business an eight out of 10. Logically, yes, but under the NPS framework, those eights are passives. You have zero promoters and zero detractors. Speaker 2: That seems harsh. Okay, so a consistently good service. Speaker 1: leaving your company with a final net promoter score of absolute zero. Speaker 2: Wow. And conversely, if that company somehow nudges every customer from an 8 to a 9, they suddenly have 100 % promoters and zero detractors. Exactly. So the score shoots from zero to a perfect 100, just based on a single point shift. That tiny incremental change causes a massive dramatic swing in the perceived health of the company. That seems like a terrible way to evaluate employee performance. Speaker 1: You are not alone in thinking that. The academic backlash against this volatility is substantial. bet. The white paper prominently cites Byron Sharp, a professor of marketing science at the University of South Australia. He did not mince words regarding NPS. called the metric based on sloppy research and thinking, ultimately concluding that it is snake oil, fake science. Speaker 2: you say. snake oil that is a serious indictment for a metric that is everywhere. Speaker 1: And furthermore, multiple award-winning longitudinal studies over the years have consistently failed to replicate Rightheld's original claims that NPS is inherently superior to other metrics for predicting actual business growth. Speaker 2: So if CSAT is too basic and NPS is mathematically volatile and maybe snake oil, that brings us to the new kid on the block introduced around 2010 CES or customer effort score. Speaker 1: Right, and the philosophy here abandons the idea of satisfaction entirely. Speaker 2: Instead, the survey asks how much effort you personally had to put forth to handle your request. Speaker 1: The theory driving CES was that companies waste too much money trying to delight customers during routine support interactions. Exactly. The creators argued that customer service experiences account for very little actual brand loyalty. Loyalty is driven by the core product itself. Speaker 2: just fix the problem, basically. That makes sense. I don't love my internet provider because the support agent was nice. I just want the internet to work. Speaker 1: However, bad service experiences absolutely drive disloyalty. The goal, therefore, is simply to make problem solving as effortless as possible. Measure the effort, reduce the friction, and you stop the disloyalty. Speaker 2: The logic sounds really sound. If I have to call my provider three times and repeat my account number to four different people, my effort is sky high and I am definitely going to cancel my service. Speaker 1: But the academic data testing that logic tells a completely different story. Really? What's fascinating here is a comprehensive statistical study from the University of Groningen put all of these metrics to the test to see which one actually predicted customer retention the most accurately across various industries. OK. They did. And CES fared the absolute worst of all the metrics tested. Wait. Speaker 2: So they looked at long-term behavior. How does the metric specifically designed to stop customer churn end up being the worst at predicting it? Speaker 1: The researchers highlighted the time dimension of these metrics as the critical flaw. CES is inherently backward looking. It asks the customer to evaluate an effort that has already been expended in the past. CES focuses on the present state. Are you satisfied with the resolution right now? NPS attempts to focus on the future. Will you advocate for us tomorrow? Speaker 2: And CES has just stuck in the past. Speaker 1: The poor performance of CES was attributed to the fact that looking backward at friction is far too narrow of a lens. It completely ignores the emotional outcome of the interaction, focusing only on the mechanical steps taken to get there. Speaker 2: So a customer might have had to put in a lot of effort to solve a complex software bug, but if the support agent was incredibly empathetic and ultimately fixed the issue, the customer might still be highly satisfied and loyal. Speaker 1: Exactly. CES would flag that as a failure while the reality is much more nuanced. Which is why relying on a single measure, regardless of which one you choose, is statistically unwise. Sure. Both academic researchers and industry practitioners advocate for combining multiple customer feedback metrics to find hidden insights that a single score would just mask completely. Speaker 2: You need the full picture. The white paper makes a brilliant analogy here, actually. Think about going to a doctor for a comprehensive checkup. A physician isn't going to just measure your blood pressure, completely ignore your BMI, ignore your blood sugar levels, and then confidently declare you perfectly healthy. Speaker 1: They require a combination of distinct metrics to understand your true holistic health. Speaker 2: Right, and the sources detail a specific method for mapping this multi-dimensional data proposed by an expert named Sarah Chambers. Yes, the chart. Exactly. She recommends plotting a customer's MPS score against their C-Asset score. And when an organization cross-references these two data points, highly distinct customer personas emerge. Advocates. Right, the advocates. These are individuals with high C-Et and high MPS. They had a great immediate experience, and they hold long-term loyalty to the brand. Speaker 1: quadrant. starting with the ad. But the real value of this matrix is finding the group she calls the wild cards. These are customers with a high CSAT, meaning they are perfectly satisfied with the support interaction they just had, but they simultaneously hold a very low NPS. Speaker 2: Wildcards. meaning they are not loyal and will actively tell others not to use your service. Speaker 1: A wild card represents a massive blind spot for a company looking at isolated data. If an executive only looked at the CSAT score, they would see a happy customer and consider the interaction a total success. Speaker 2: But by combining the data, they realize this person is actually a severe churn risk. The support agent may have been polite and efficient, but the customer fundamentally dislikes the core product or the pricing model. Speaker 1: Exactly. And on the completely opposite end of the spectrum, you have the Wasteland. Sounds bleak. It is. These are the customers returning low CS-SAT and low NPS scores. Those are the relationships that are likely already unsalvageable. Evaluating these multiple dimensions dramatically improves how accurately a company can predict retention. Speaker 2: It all hinges on one very fragile assumption, doesn't it? That the data collected from these surveys is actually accurate and representative in the first place. Speaker 1: It does. And that brings us to the most systemic problem facing the industry today, survey fatigue. Consumers are currently drowning in what the white paper literally describes as a blizzard of customer satisfaction surveys. Speaker 2: The fatigue is just unavoidable. Every time you buy a coffee, order a pair of shoes, or download an app, you get an email begging for a rating. It never ends. And because of that constant bombardment, post-task surveys currently achieve an incredibly dismal 4 to 7 percent response rate. Speaker 1: for ever. When you are operating with response rates in the single digits, you run into severe statistical flaws, specifically regarding the margin of error. Right. If a company has 10,000 interactions a Wnt and only 4 % respond, they are basing their entire strategic direction on the opinions of just 400 people. And those 400 people are rarely a statistically representative sample of the broader customer base. Speaker 2: because the sample size is just too small. Let's look at the specific biases corrupting that small sample size. I saw the researchers mention survival bias regarding these surveys. Now, A usually associate that term with evolutionary biology or historical data analysis, not customer service. What are they getting at here? Speaker 1: Survival bias in this context means your surveys are only capturing the sentiment of customers who are still surviving within your ecosystem. Speaker 2: Meaning they are still actively doing business with you. Speaker 1: Exactly. The survey infrastructure completely misses the ex-customers, the ones who grew so frustrated they already abandoned the company. An organization ends up measuring the happiness of the people who stayed, completely blind to the systemic issues that drove the others away. Speaker 2: That is a massive blind spot. There was also volunteer bias, which the researchers refer to as the mum effect. And this actually goes back to my experience trying to create a podcast. so? Well, when I first launched my show, the only people who took the time to leave a review were my closest friends who just wanted to be supportive or the occasional person who absolutely hated my microphone audio and wanted to complain. Exactly. The thousands of people in the middle who thought the show was just fine never bothered to leave a rating. Speaker 1: The extremes. And that is exactly the mum effect. It refers specifically to the psychological tendency where customers are generally hesitant to provide negative feedback directly to a person or company. Rather than leaving a bad score, they simply choose to say nothing and take their business elsewhere. Speaker 2: So you end up capturing the extreme vocal highs and the extreme angry lows, leaving a massive void where the average customer experience actually lives. Speaker 1: Precisely. And if you operate in B2B business to business, you encounter the wrong audience problem. A massive enterprise buys a new software platform. The executive signs the check, but the everyday employees are the ones who actually have to use the software. When the NPS survey gets sent out asking, would you recommend this product, it goes to the executive's inbox. The person rating the software has absolutely no idea how difficult it is for their team to use on a daily basis. Speaker 2: ⁓ this one is fascinating. So the data becomes hopelessly skewed. And because corporate leadership ties so much weight, and often direct financial compensation, to these flawed numbers, it inevitably leads to the most predictable human behavior of all. Gaming the system. Speaker 1: The ways employees manipulate these surveys is both fascinating and a little dark. Speaker 2: It really is. The white paper quotes user experience expert Jared ⁓ Spool on this phenomenon. He lays it out perfectly. He says, if a manager's bonus is tied to an increase in MPS ratings, offering a customer $100 incentive is a highly effective way to raise those scores. Speaker 1: It is an inconvenient truth for the industry. When an agent's livelihood depends on a metric, they will optimize for the metric, not the customer experience. Right, support staff will figure out ways to solicit feedback only from the customers they know are thrilled, while subtly closing out tickets for frustrated customers in a way that avoids triggering a survey email entirely. Speaker 2: Just basic incentives. The text even highlights user interface tricks, like companies deliberately designing their mobile app surveys to clip the negative numbers off the edge of the screen. ⁓ Forcing the user to physically swipe to see the one or the two, while the four and the five are easily tappable right there. The data is entirely corrupted by the time it reaches the boardroom. But here's where it gets really interesting. Even the architect of the most famous metric agrees the system is broken. Speaker 1: Just completely hiding them. The admission from Frederick Reichelt, the creator of the Net Promoter Score, is perhaps the most striking detail in the entire white paper. Speaker 2: Reichel did an interview with Bloomberg in 2016, and he openly admitted that he is completely sick of surveys. He expressed deep frustration over the ubiquitous survey fatigue and criticized how companies pressure customers to give perfect scores, effectively rendering his own creation totally meaningless. His exact quote from the interview was, the instant we have a technology to minimize surveys, I'm the first one on that bandwagon. Speaker 1: He really did. When the creator of the ultimate survey metric openly longs for a technology to minimize surveys, it signifies that the current methodology is collapsing under its own weight. It forces the industry to look for a completely new paradigm. Speaker 2: So what does this all mean? How does a business fix a broken system where the data is biased, the response rates are hovering near zero, and the customers are too fatigued to even click a button? Speaker 1: The solution presented in the SupportLogic white paper represents a massive pivot in how we think about feedback. The industry must stop relying on explicit feedback, where a company stops the customer journey to demand a rating and start gathering implicit feedback. Speaker 2: Right, implicit feedback. And this relies heavily on artificial intelligence and natural language processing, or NLP. Speaker 1: Exactly. Instead of sending a survey after a support interaction has ended, the company uses AI to analyze the raw content of the interaction itself. Speaker 2: It reads the text of the support tickets, the back and forth emails, and the transcripts of the support calls. Speaker 1: The technological leap here is profound. Natural language processing algorithms do not simply scan a transcript looking for the word angry. They extract fine-grained sentiment and urgency by analyzing the entire context of the communication. The AI evaluates sentence structure, the use of capitalization and punctuation, the frequency of replies, and specific phrases that indicate escalating frustration, like a customer saying, I have explained this three times already. Speaker 2: That is wild. So the software is reading a support ticket and calculating the customer sentiment mathematically, completely eliminating the need for the customer to click a one out of five button. Speaker 1: But by analyzing the language implicitly, organizations solve multiple systemic flaws simultaneously. First, they completely eliminate survey timing issues. Analysts no longer have to worry if they are sending the survey too soon after an interaction or too late because the AI is evaluating the entire continuous journey from the first email to the final resolution. Speaker 2: and it totally bypasses the blizzard of survey fatigue. customer never has to do any extra work, they just communicate naturally and the system understands how they feel. Speaker 1: Furthermore, the statistical margin of error disappears. Because the AI analyzes every single ticket and transcript, the response rate effectively goes from 4 % to 100%. Exactly. The company captures the sentiment of the silent majority. There is no volunteer bias. There is no survival bias. And crucially, it becomes impossible for employees to game the system. An agent cannot click negative scores off a screen when an AI is reading the raw, unedited transcript of a frustrated client. Speaker 2: You capture everyone. From a practical business standpoint, the most actionable benefit is the shift in timing. A traditional survey is essentially a postmortem. It tells a company that a customer was unhappy two weeks ago, long after they've already canceled their subscription. Implicit feedback is continuous and real-time. Speaker 1: Moving from a reactive stance to a proactive stance is the ultimate goal of NLP integration. If the AI flags an ongoing email thread as highly urgent and negative while the ticket is still open, it allows management to intervene, escalate the issue, and course-correct the experience before the customer turns. Implicitly derived metrics offer a fundamentally more accurate, holistic, and actionable understanding of the true customer experience. Speaker 2: That is incredibly powerful. To summarize the journey we've taken today, CSAT, NPS, and CES all served a purpose in the evolution of the service economy. There were early attempts to quantify the incredibly complex nature of human loyalty, but the traditional explicit survey is dying, suffocated by its own statistical biases, the gaming of the system, and massive customer fatigue. The future is no longer about asking people how they feel on a scale of one to 10. The future is AI-driven, Speaker 1: It really is. Speaker 2: implicit feedback that understands how they feel based on how they naturally communicate. Speaker 1: It is a technological leap that solves the core measurement problems of the past 20 years. However, relying on algorithms to interpret our emotions inevitably leaves us with a rather profound philosophical question as we look forward. Let's hear it. This raises an important question. If artificial intelligence can now accurately read our true feelings, our latent frustrations, and our baseline loyalty just by analyzing the words and cadence we use in our daily emails and chat logs, without us ever consciously opting in to fill out a survey, what does that mean for our digital privacy? Speaker 2: That is a big question. Speaker 1: Will the future of customer service feel incredibly intuitive and magical? Or will it feel like our every emotional reaction is being quietly monitored, analyzed, and scored by a hidden algorithm? Speaker 2: Magical intuition versus an invisible emotional scorecard. It's a fascinating thought to leave off on and something to seriously consider the next time you type out a frustrated email to a support desk. Thank you so much for joining us on this Deep Dive today. We hope you walk away looking at every one to 10 scale a little bit differently. Keep questioning the data around you and we will catch you next time.