Measurement|survey manipulation|survey results

I’ve been in the automotive industry for some time and concerns about dealership attempts to interfere with the customer satisfaction measurement process have been around for a long time as well, but lately they seem to be intensifying. Do you know how to guard against survey manipulation? My hope is this blog will give you some things to think about, start a conversation, and help you understand my perspective on how manufacturers can work towards addressing survey manipulation.

Here’s what I’m seeing. The first step most manufacturers and suppliers take is to put processes in place to identify dealerships that attempt to manipulate survey results. While this step is extremely important it only addresses part of the problem. To effectively address this issue, manufacturers, dealers, and customer satisfaction measurement suppliers need to work together.  And how they do that is by:

1. Establishing, communicating, and consistently enforcing consequences and a strong anti-manipulation policy
2. Designing survey systems that make manipulation of survey results difficult
3. Setting up systems and processes to identify dealerships that attempt to manipulate survey results
4. Defining acceptable practices and working with dealerships to implement them
5. Designing reward and compensation programs that minimize the motivation to manipulate customer experience measures 

Establishing Consequences

Consequences of engaging in unacceptable practices need to be specified in advance and communicated to dealerships. These consequences need to be enforced if dealerships are identified as engaging in survey manipulation. Manufacturers will need to utilize consequences which they have the ability to enforce. Effective consequences include:

• Assigning the lowest score to manipulated surveys
• Requiring dealerships to reimburse the manufacturer for the cost of conducting their CSI surveys for the time period under which manipulation occurred
• Denying customer satisfaction-based compensation or rewards to dealerships that have manipulated their scores
• Subjecting dealerships that have been identified as manipulating their scores to audits of all or many of their manufacturer programs
• Including language in the manufacturer/dealer franchise agreement that customer satisfaction survey manipulation is grounds for removal of the franchise 

Making Manipulation Difficult

Manufacturers and their customer experience suppliers should design systems that make survey manipulation as difficult as possible. These systems need to be continually monitored and updated to address new methods of survey manipulation.

Some ways to make survey manipulation difficult include:

• Use contact information that can be verified
• Use manufacturer databases, not dealership management systems, as the source of sales and service samples
• Don’t conduct point-of-purchase/point-of-service surveys
• Use contact information that can reach all, or almost all, customers
• Use multiple contact methodologies
• Allow customers the option of remaining anonymous

Designing Processes to Reduce Manipulation

The less likely dealerships are to get caught manipulating survey results, the more tempting it will be for some of them to try. Therefore, processes need to be put in place to identify potential survey manipulation. These processes will be specific to the given project and the methodologies used, but can be broadly categorized as follows:

• Examine customer contact records for suspicious mail or email addresses and phone numbers
• Examine incoming materials including IP addresses
• Examine customer comments
• Examine contact resolution reports
• Examine the actual data
• Randomly audit survey responses
• Include a survey manipulation question in the survey

Implementing Acceptable Practices

There are many practices dealerships can implement that both increase customer satisfaction scores and improve the customer experience. If manufacturers embrace these practices and work with dealerships to implement them, dealerships will have less need to engage in unacceptable survey manipulation. Some practices I have seen encouraged by manufacturers include:

• Showing a blank survey to all customers and requesting that they fill it out honestly and return it
• Asking customers if they are unsatisfied about anything regarding their experience and attempting to resolve the issue
• Telephoning customers within a few days of a sales or service event, inquiring about their satisfaction, and engaging in appropriate efforts to resolve any dissatisfaction
• Explaining the importance of receiving customer feedback for both the dealership and the manufacturer

Minimizing the Motivation to Manipulate

While it is important to hold dealerships accountable for their treatment of customers, and compensation based on customer satisfaction scores is the most obvious way to do that, I believe the way in which some reward and compensation programs have been designed has exacerbated the problem of dealerships attempting to manipulate the system. Some suggestions for setting up programs that minimize the motivation to manipulate results while still holding dealerships accountable for customer satisfaction are:

• Compensate/Reward based on several desired business metrics of which customer satisfaction is only one.
• Use a tiered compensation strategy rather than an “all
or nothing” strategy.
• Separate the dealership performance appraisal process from the dealership diagnostic process.
• Require large sample sizes for determining scores.
• Keep monetary rewards at a reasonable level so they remain rewards and do not turn into business necessities.
• Consider non-monetary rewards

While many of these concepts and steps to reduce survey manipulation are all being done by most manufacturers, few are doing all of them together.

Don’t Forget You May Be Different

Survey manipulation is a systemic issue, but it affects vehicle manufacturers differently. Each manufacturer, with input from its dealerships, needs to decide what practices are acceptable and unacceptable. Each manufacturer needs to determine how seriously it wants to pursue this issue and how serious consequences should be to its dealerships who engage in unacceptable practices. This blog offers some guidelines and ideas manufacturers may want to consider when making these decisions. I’m always open to answer questions and I would enjoy hearing what you’re doing or your thoughts about survey manipulation, feel free to join this conversation on CX Café or check our webinar on survey manipulation.