Klement’s Fallacy Misleads The Jobs-to-be-Done Community
Conflating customer preferences with desired outcomes will take companies back to the dark ages of unpredictable innovation.
In Alan Klement’s article, The Illusion of Measuring What Customers Want, Klement claims: “customer preferences and desired outcomes cannot be quantified in a reliable and valid way.” He feels so strongly about his assertion that he goes on to say “If someone claims they have a method that promises to measure (quantify) what customers want, they’re trying to scam you.”
That is a pretty bold accusation. One, I will argue, that is built on fallacy — a straw-man argument. In Klement’s article, he purposely conflates customer preferences and desired outcomes and tries to convince the reader they are one in the same thing, when in fact they are not. He explains that customer preferences cannot be quantified in a reliable and valid way (an argument that has its merits) and asserts the same is true for desired outcomes — a false conclusion that is based on the fallacy he purposely introduces.
A customer preference and a customer desired outcome statement are very different, each with its own unique definition and characteristics.
The term “customer preference” is defined as “the type of product an individual customer likes and dislikes.” It refers to a solution: a preference for a product or possibly a feature. It is well established that asking customers about their solution preferences — what features or solutions they prefer — does not lead to predictable innovation as Klement suggests. This is why the research we conduct as part of our Outcome-Driven Innovation (ODI) process doesn’t quantify customer preferences. Instead it is focused on what we call the customer’s desired outcomes: the metrics that customers use to measure success when getting a core functional “job” done.
The desired outcome statement, which my team and I invented and named over 25 years ago, is a specially constructed need statement that has a unique set of characteristics: desired outcomes are devoid of solutions, stable over time, measureable, controllable, structured for reliable prioritization in a quantitative customer survey, and tied to the underlying process (the job) the customer is trying to get done (see Inventing the Perfect Customer Need Statement and Giving Customers a Fair Hearing).
Clearly, a desired outcome is not a customer preference — a desired outcome describes a need, e.g., minimize the likelihood that the music sounds distorted when played at high volume. On the other hand, a customer preference relates to a preferred solution, e.g., the Bose SoundTouch speaker versus the Sonos PLAY:5 speaker. People prefer and buy solutions that address their underserved outcomes.
While companies often conduct research that asks respondents to choose between solutions, our ODI-based research does not ask customers to choose between solutions or outcomes. ODI research is not about choices and preferences as Klement falsely suggests in his Tweet below. To the contrary, it is designed to determine which outcomes are unmet in the customer population (important but not well satisfied) and to what degree. ODI-based research may reveal, for example, that 15 out of 100 desired outcomes are important and unsatisfied. Customers aren’t asked to choose between them. Developers are tasked to address all of them. With this knowledge, developers (not the customer) can correctly choose which of the 100 outcomes to address to ensure the success of their product. This thinking is fundamental to the practice of Jobs-to-be-Done Theory and ODI.
As we have proven over the years, it is possible to successfully quantify unmet customer outcomes. Our outcome-based research methods, which overcome the issues typically involved in quantifying customer preferences, are used in dozens of Fortune 100 companies and consistently reveal underserved (and overserved) desired outcomes (see Turn Customer Input into Innovation, HBR Janury 2002, for our first published case study). See Quantifying Unmet Customer Needs for details on how the ODI methodology incorporates dimensionless quantities into the Opportunity Algorithm (another part of the ODI process that Klement likes to attack with straw-man arguments).
I could argue that Klement’s fallacy was committed unintentionally, and that he honestly does not understand the difference between customer preferences and customer desired outcomes. This is a possibility as Klement often conflates solutions with outcomes in the examples he uses.
In one of his articles Klement argues that “any metric tied to a desired outcome or a preference (note the conflation) is always in flux and susceptible to outside influences”. He attempts to support his argument by saying:
“For example, you’re at a restaurant with some friends. The waitress comes over to you. “I’ll take the steak with mashed potatoes” you say. Then your friend orders, “I’ll have the steak with grilled vegetables”. Upon hearing your friend, you decide to change your order. “Actually, instead of the mashed potatoes I’ll have the grilled vegetables as well.” Even though you knew grilled vegetables were on the menu, you first chose mashed potatoes. But for some reason, after hearing your friend order vegetables, you switched. This happens all the time. So often in fact, behavioral economists call it a Preference Reversal.”
Yes — it happens all the time — people change their minds as to what side dish (solution) they want. To be clear, in Klement’s example, mashed potatoes and grilled vegetables are solution preferences, not desired outcomes. Klement conflates a “desired outcome” with a “preference” up front then uses a solution-based example to try and convince the reader that it also applies to a desired outcome — but it does not. It’s a fallacy. A desired outcome in that situation might be, “minimize the likelihood that a meal will result in weight gain,” a statement that is not subject to Preference Reversal.
In another article authored by Klement he states:
“It’s also a mistake to believe that customers know what they want — or that we should study customer “needs”. In fact, customers don’t know what they want or what they need.”
Here again Klement conflates solutions with needs. Knowing what solutions you “want or need” doesn’t make the solution a need. Solutions and needs are NOT one in the same — and conflating the two takes the innovation process back to the dark ages.
I could also argue that Klement’s fallacy was committed intentionally to sow seeds of doubt regarding ODI and to advance his own agenda. That may sound harsh, but there is certainly enough evidence to support the possibility of intentional deception.
For example, in response to the argument I make above, Klement doubles down on his fallacy in a Twitter exchange by saying:
“In behavioral economics, “preference” = “what the customer wants / likes”. If a customer “desires” an outcome, that is a preference.”
Notice what Klement does: he purposely conflates “desires an outcome” (the act of desiring something, like a solution) with a “desired outcome statement”— a term I define specifically as “a metric that customers use to measure success when getting a job done.” It is correct to say that if a customer desires a solution, that is a preference. But it doesn’t make sense for Klement to say, “If a customer desires an outcome, that is a preference,” unless he is intentionally redefining the term “desired outcome” to try and manipulate or persuade people by deception. Klement is trying to redefine the term “desired outcome” and claim he has a valid argument against ODI: this is intellectual dishonesty.
This would not be the first time Klement has made deceptive claims regarding ODI and JTBD. There is a pattern of deception.
As the Twitter posts below show, Klement falsely claims that ODI and JTBD are utterly different. He misrepresents ODI as something it is not. He insinuates I am a latecomer to JTBD, not a pioneer. He insults Clayton Christensen, calling him “an IYI (intellectual yet idiot)”. He calls me a fraud, claiming I’ve retrofitted ODI to JTBD when in fact I introduced ODI and the underlying theory to Christensen who popularized it under the JTBD label. He claims he wrote the first book on Jobs-to-be-Done in 2016, while he continually mocks What Customers Want, the actual first book on the subject published back in 2005.
On top of that, Klement has also gone on to attack other parts of the ODI process in the same uniformed manner he used to create this fallacy. I debunked his attack on the Opportunity Algorithm here.
I’ve spent a lot of years working to advance the innovation process. I can understand and appreciate somebody trying to compete, get ahead, contribute and make a difference. But it is best done with intellectual honesty, morality and arguments based on fact — and it deserves a call out when it is not.
Learn more: download a FREE PDF version of my latest Jobs-To-Be-Done book, JOBS TO BE DONE: Theory to Practice.
