Customer Experience Measurement – Direct v/s Indirect

In my last blog, we discussed about three key questions an organization needs to answer to bring in clarity in their CX improvement program as well as assessing the benefits from the same. These are,

1. Should Customer experience be measured as stand-alone value or it should be measured in terms of “rate of change” in the CX measure?
2. Should we measure CX using Direct Customer Feedback or through Indirect (Derived) feedback?
3. Should we use past data or future (predicted) data?

We deliberated on whether we should measure the CX in a stand-alone manner or using “rate of change” in my earlier blog and concluded that measuring the “rate of change” will benefit the organization more. It will also help to assess the stage of the CX lifecycle the initiative is passing through and take the appropriate action. You can read the same using this URL--  https://personalandprofessionalexcellence.blogspot.com/2019/11/customer-experience-measurement-stand.html

In this blog, I am going to discuss the second Question i.e. Should we measure CX through Direct Customer Feedback or through Indirect (derived) feedback? First let us understand what we mean by Direct as well as Indirect

Direct Feedback: When a feedback about customer experience is obtained directly from a customer through any kind of Q&A mechanism, it is called a direct feedback. The mechanism could be a physical survey, online survey, meet & greet meetings, telephone, home visits, etc.  We ask customer to rate the product or service and their rating decides the CX level.

Indirect (Derived) Feedback: Indirect Customer feedback is calculated by establishing a relationship between multiple parameters, other than answers from customer and available through customer transactions. Multiple parameters like sale of a specific product before and after the initiative is launched or changes in the traffic on website or customer behavior on website or customer response in various geographies, etc. which are available with the organization can be used to create an algorithm which could represent the pattern of CX behavior and used for measuring the status of CX at any point of time. 

Following table provides a comparison between the two methods on certain criteria which can throw some light on pros and cons of both the methods

	Direct Feedback	Indirect (Derived) Feedback
Customer Involvement	Real feedback through multiple means of interviewing / Survey	Feedback interpretation depends on the definition of relationship between the parameters measured and customer experience. Experimentation is required to build the relationship between multiple parameters and CX
Ease of gathering information	Process of gathering & analyzing feedback is predominantly manual or semi-automatic. Thus, requires longer time	No special information gathering is required. Existing information is used. So, process could be completely automatic
Frequency for measurement of CX	Limitation on how frequently you can go to a customer and reluctance of customer to give feedback frequently.	No limitations and every transaction could be used for calculating CX. The measurement could be near real time.
Coverage – Customers	Due to inherent nature of the process, there is a limitation on how many customers could be contacted and how many really respond. Many times unhappy customers voice their feedback and happy customer do not respond creating incorrect picture.	No limitations, feedback calculation can consider every transaction
Coverage – Bias in selection of respondents	Due to limitations on reaching customers, there is a possibility of introducing a bias in selection of customers to provide feedback and thus influencing the CX	No bias as entire data can be used to give real picture.
Coverage – Time period	Due to longer frequency, the information gathered does not represent the experience for the whole period but typically for the events / transactions just before the survey.	As the data is used continuously, a true overall picture for the respective time period can be achieved
Coverage - Customers / Non-Customers	Typically, the customers who have bought products / services are part of this exercise and customers who have not bought the products / services are not. As a result, we tend to miss out on the experience traits which have made some customer to go away from our products / services	The indirect mode enables us to compile data for those who have not bought the product / services through some of the parameters like "Returns", "Selection of items and removing from shopping cart without a purchase", "Complaints", "Replacements", "footfalls v/s sales", "termination of services", etc. This information is available and could be used for building CX model.
Response time for CX feedback	Understanding CX, deciding action based on it, implementing the same and measuring the impact is a very long process and could easily take 2-3 quarters if not more. It takes time to detect negative trend.	The CX could be measured on a continuous basis. The frequency could be daily or better depending on the algorithm and data collection & crunching capability. Company can get the trend in CX as it is taking place and empowers it to tweak the CX intervention based on the same real time. Negative trend could be tracked early.

Based on the above table, it is evident that indirect method wins the game. Only challenge here is to get the algorithm right. Unless we get the algorithm right, all the measurement is of no use and can lead us to wrong direction. So if the organization can use new techniques in AI/ML to create a self learning algorithm using the past data along with experimentation; organization can really go ahead of curve in continuously improving CX.

Ref :    

•  CX Life cycle management for continuous improvement in Customer Experience and Revenue/Customer https://personalandprofessionalexcellence.blogspot.com/2019/07/cx-life-cycle-management-for-continuous.html
• AI for early detection of Customer Experience Fatigue and effective extension of CX lifecycle https://personalandprofessionalexcellence.blogspot.com/2019/08/ai-for-early-detection-of-customer.html

 

Customer Experience Measurement – Stand Alone v/s Rate of Change

Customer has become king again and is in a position to demand much more beyond better pricing and service from sellers. Sellers are also obliging in multiple unique ways to differentiate themselves in the eyes of customer. Several experiments are being made and significant dollars are set aside for this purpose. The ultimate motive of this push is to improve revenue from existing streams as well as add new revenue streams for the future. The success of the initiatives is measured in terms of how much the needle has moved.

There are two steps in this process; first step is, to define how we would measure customer experience and second step is, how do we connect this CX measure to Revenue movement. This kind of rigor is relatively new to CX and multiple options are being tried by organizations to connect it to revenue. It is not perfected yet, however learning from each experiment is improving the clarity.

An organization will need to answer these three questions to bring in clarity in their CX program as well as measuring benefits from the same.

1. Should Customer experience be measured as stand alone value or it should be measured in terms of rate of change in the CX measure?
2. Should we measure using Direct Customer Feedback or through Indirect (derived) feedback?
3. Should we use past data or future (predicted) data?

I have analyzed first question in this blog and identified possible way forward for companies.

Stand Alone or Rate of Change:

There are a few popular metrics to measure CX today; prominent being NPS and CSAT. Companies have devised ways to capture the NPS / CSAT score at definite intervals and use the feedback from each survey to create next round of initiatives. Some of the companies have built the necessary rigor in to their processes and collect the measurement at regular frequency. However, they still face some challenges like,

• The score is influenced many times by specific performance just before the survey and does not represent the entire period between two surveys 
• The highest frequency of such survey is Quarterly, but Half yearly or yearly is preferred due to investment of time and efforts in executing one cycle for company and customer
• Chances of bias in selection of respondents. Detractors or customers who did not make a purchase are missed out depriving the comprehensive CX level.

These challenges create limitations in terms of how fast one can respond, how comprehensively one can understand the trend as well as how to measure the actual impact on revenue movement.

If we want to understand the customer experience, how it is getting impacted by a specific initiative as well as how it is impacting revenue with minimal delay, observation of Rate of Change in CX is the answer. It provides us with the direction of impact and extent of impact together, which is critical to take the next course of action. Knowledge about success or failure of the initiative helps organization to make a kill or stop the losses, so faster is better. If the trend of CX is positive and rate of change is increasing, the initiative is working and impact is increasing exponentially. If the rate of change is zero, initiative has no impact on CX and mostly will not have any impact on revenue. Declining rate of change signifies decline in impact and possible revenue reduction.

The model in the adjacent figure depicts the status and revenue impact of a CX initiative through out its lifecycle. Typically, a positive burst in terms of increasing CX is seen at the time of introduction of new initiative and the relative impact reduces as the initiative matures. Slowly the impact is nullified and then ventures into negative territory. Once we map our initiative to the quadrant, we can determine our next steps.

Success of this model depends on frequency of capturing CX measure. Faster the capture, better the results.

With the current metrics used i.e. NPS or CSAT, it is almost impossible to capture the customer feedback at higher frequency due to the nature and mechanism of executing such initiatives. However, with the success of experimentation with AI/ML, certain algorithm-based metrics could be created using the data on inputs and outputs available with the company. This metric value could be calculated on daily basis or segment basis or geo basis to see how it is moving to plot it in the right quadrant.

Ref :    

• 1CX Life cycle management for continuous improvement in Customer Experience and Revenue/Customer https://personalandprofessionalexcellence.blogspot.com/2019/07/cx-life-cycle-management-for-continuous.html
• AAI for early detection of Customer Experience Fatigue and effective extension of CX lifecycle https://personalandprofessionalexcellence.blogspot.com/2019/08/ai-for-early-detection-of-customer.html