On the surface, it seems like an obvious application for a ChatGPT customer experience approach. A survey is pretty straightforward, correct? Not so fast.

Keep reading to find out what happened when I tested this approach and why it may not be the best way to go when it comes to your customer listening approach.

Testing ChatGPT for Customer Experience Questions

I started off with a simple question for ChatGPT, hoping for a simple customer satisfaction survey, typing in, “Write me a survey.” You can see the screenshot of the output below.

After reviewing the generated answer, you may be asking, “what’s missing?” Well, to the untrained eye, there could be little to no difference between a traditionally written survey and a ChatGPT customer experience survey. After all, there are demographic questions, the typical “How satisfied were you with your experience,” and other basic survey asks.

But here is what stands out to me as a glaring absence. What is missing is pretty much the most important part of any survey: the link to the business questions you are trying to answer by launching a survey in the first place!

Quick PSA from Jim: Creating surveys is an important topic,  but I would be remiss if I didn’t mention that while surveys are a tried-and-true method of collecting customer feedback, they are not the only way (or the best way, in many cases) to hear from customers. With so many channels available for you to monitor the voice of the customer, to restrict yourself to surveys alone is to limit your insights. This is another topic for another day (but if you’re interested, you can learn more about other listening channels here). End of PSA.

 For now, let’s talk about the risks of using AI like ChatGPT to write surveys!

ChatGPT Customer Experience Risks & Best Practices You Need to Know

ChatGPT Customer Experience Questions Miss the Point

Let me ask you a question: Is the point of your CX program to launch surveys? Now, many of you are likely rolling your eyes at me, but I promise, there’s a point to this. Hopefully, you answered no. Because the point of customer experience is not to ask questions, but to listen to customers and the market to help guide your path to achieving business goals. The questions are simply a vehicle to gain insight into what will help or hinder your business on the way to realizing those  goals.

When you look at the output of ChatGPT customer experience questions in the screenshot above, these questions really miss the point. Yes, they are generic questions that we have all likely seen in surveys before, but what are they getting at? The only results I can see this survey gleaning is a scoreboard metric and some customer demographics that we might already have access to via other data sources. 

When you craft surveys, the first questions you should ask should be for you and your team. Do you have a set of northstar goals (GOALS not scores!) for your customer experience program already? Great! If not, start that conversation with your executive stakeholders and team. Only then can you truly design your program, surveys, and other initiatives with the end goal in mind. 

Once you have agreed upon a desired end goal, then you need to ask:

• What are we hoping to learn?
• Who are we hoping to learn from?
• Do we already have access to this data?

If you want to gut-check your surveys, you can check out this CX survey assessment my colleagues developed to help you optimize your surveys!

ChatGPT Doesn’t Know Your Customers Like You Do

Context is everything. And when it comes to ChatGPT customer experience questions, they won’t have any of the contextual data that you do. If your CX program has been around for a while, you likely have a mountain of customer data around. And that existing data will shape what you already know, and what questions you still need to ask. 

(Additionally, you might be tempted to feed ChatGPT some of your customer data, but that can unearth a whole boatload of security complications. Do you really want every ChatGPT user having access to your customer data? Didn’t think so.) 

An effective customer listening strategy is personal and targeted. Speaking to the customer in their language is critical. Many brands have worked hard to develop a brand persona. Asking customers for feedback in a sterile, canned voice will not yield the best results or further endear your brand to your customers. I don’t believe your brand personalization  can be accomplished by a ChatGPT survey—at least not today.

ChatGPT Is a Starting Block, Not the Finish Line

Now you may be thinking, “Jim, you’ve made a good case for the risks of using ChatGPT for customer experience surveys. But there has to be some way I can use it.” I’m glad you asked and yes, there is! 

I know we have all heard the fear-mongering conversations about AI taking jobs. And if we’re being realistic, AI will eliminate some jobs, but it will also create new ones. Those who will be safe from that chopping block are those professionals who learn how to leverage AI to increase efficiency and  perfect skills that AI alone just can’t manage without human input.

In the customer experience space, this could be leveraging ChatGPT as a starting point, then leveraging the additional context you have about your customers and your brand’s identity to perfect its suggestions. 

For example, ChatGPT can give you phrasing ideas for your survey questions as long as you are very specific in your prompts. It can also help you to think of other ways to ask questions you’ve been posing to customers for a long time, giving your same old relationship and post-transaction surveys a refresh. 

It’s not about AI or humans. It’s about humans using AI to improve and become more imaginative and efficient.

I will end with this. I do not want to come off as a “debbie downer” or, even worse, as naive. AI is going to have an increased role in customer experience and in creating the listening posts that practitioners create to capture customer insight. But, I believe true value will be well beyond simply crafting a survey. 

The real power of ChatGPT and other AI tools will be to help understand the data that comes from a survey or the multiple direct and indirect data sources that make up the voice of the customer. And, just to validate this statement, I asked ChatGPT why the voice of the customer is important? In this case, ChatGPT was spot on:

I think we can all agree that ChatGPT is right on target with that answer.