jim kalbach

The Case Against Quantitative Customer Experience Data


Most customer experience data strategies include two types of insights: qualitative and quantitative. Qualitative data tells the stories behind the customers, and quantitative data shares trackable metrics of customers’ actions and a brand’s performance.

But according to Jim Kalbach, author of The Jobs To Be Done Playbook, modern CX teams have an over-reliance on quantitative data, which can distract from building strong customer relationships.

When brands have a question about a program’s effectiveness or how customers feel about a certain trend, they go straight to the quantitative metrics. But those metrics don’t often reflect the real world. Kalbach says that brands must talk to customers to better interpret quantitative data. Listening to customer insights gives context to the numbers and paints a more accurate story. 

Quantitative data is reliable, meaning it can be repeated to get the same results. Every time a company tracks NPS, for example, it knows how the metric will be measured and what the results will mean. That reliability makes quantitative data desirable for CX teams and executives, but it doesn’t necessarily mean the data is completely valid on its own. 

Many brands dismiss qualitative data saying that the sample is too small and the insights are too anecdotal. But Kalbach believes listening to individuals is how brands can gain understanding to interpret the quantitative data. Instead of jumping straight into large-scale surveys, Kalbach says executive and customer experience teams need to be talking to customers and sharing their feedback.

It’s often the difference between thick and thin data, a term coined by Tricia Wang. Thin data says one thing, like how many people downloaded a product or converted. It can be accurate, but it shows a singular action. Thick data measures how people feel before, during, and after that action and gives more context to the data. Because it is more involved, the sample size is smaller, but it provides a thick description of the human experience. Kalbach says brands need both to know the whole story of their customers.

Quantitative data doesn’t replace qualitative data, but it does complement it. For that reason, Kalbach says brands still need quality quantitative data. The metrics might not be inaccurate, but they don’t necessarily tell the whole picture. Qualitative data provides a rich, emotional, and human aspect to interpret the raw metrics. 

Brands that rely on quantitative data only have half the story of their customers. Talk to customers and expand your qualitative data to get the full picture and excel in customer relationships.


Blake Morgan is a customer experience futurist and the bestselling author of The Customer of the FutureFor regular updates on customer experience, sign up for her weekly newsletter here

Share this post