Auto-Analyzing Sentiment in Survey Feedback Using NLP: How Do Customers Feel About Your People?

Wootric (now Pearl-Plaza) uses CX metrics—Net Promoter Score, Customer Satisfaction and Customer Effort Score—to monitor customer experience for high-growth companies. We take a customer-centric approach to survey design. For example, our modern 2-question Net Promoter Score survey invites customers to elaborate freely on the reason for their score. We deliver millions of surveys that achieve response rates of 30-40%, generating thousands of pieces of unstructured customer survey feedback each week.

Why Is Survey Feedback Important?

Because when you communicate directly with your customers, they can identify exactly what works, what doesn’t work, and where the pain points are that may be detracting from their experience. Honest feedback gives you the insights you need to make improved business decisions and optimize the customer experience. As such, the right customer survey can play a significant role in increasing customer retention and helping your organization reach its goals.

Customer feedback comments are a treasure trove of information that can help a company shape their product and service for success. Until now it has been difficult for a Customer Insights Manager or customer experience management (CXM) teams to mine and aggregate qualitative data for insights that can guide business decisions.  

Auto-Tagging with Sentiment Analysis

We recently announced early access to a new product feature: auto-tagging. For auto-tagging, we use our homegrown machine learning system along with Google Cloud Natural Language API to automatically categorize open-ended customer-survey feedback that our customers get as part of their NPS, CSAT and CES programs. The goal is to help companies put some structure to all of this qualitative data. We have a long list of customers eagerly waiting to get their hands on this feature. It’s a good problem to have.

In addition, we are developing the ability to identify the sentiment of the feedback. The goal is to determine not only what the customer was talking about, but to say whether the feedback is positive, neutral or negative. It is particularly complex to decipher multiple “sentiments” within a single comment.  

Here is an example feedback comment that we received in response to a Net Promoter Score survey on our own production application (we practice what we preach):

“Setup guide for customizing social sharing on iOS SDK was confusing. Diego reached out with sample code which helped a lot.”

Wootric (now Pearl-Plaza) is a SaaS product, so our auto-tagger uses a SaaS data training model and applies three tags to this survey response (Documentation, SDK, People), and assigns a NEUTRAL sentiment for the feedback as a whole. This obviously is pretty good, but we want to do more.

Wouldn’t it be nice if we could dig deeper into survey feedback and apply sentiment for each tag as well? In the above example, the customer was not happy with the SDK Set-up Guide, but was pleased with Diego’s assistance.  This nuance is buried under the overall NEUTRAL sentiment. Ideally, the Documentation and SDK tags would be identified as having negative sentiment, while the People tag would be positive.  

We Can Identify Sentiment Associated with People, Team, Organization or Location

Buried survey feedback is not a trivial problem to solve. However, using Google Cloud Natural Language API’s latest feature called “entity sentiment analysis” we have made progress. We can already get sentiment for entities referenced in feedback where an entity is defined as People, Team, Organization and Location. In this case, Diego is an entity of type People and positive sentiment is correctly attached to it.

Example of Auto-tagging an Pearl-Plaza NPS Survey Response

A Business Use Case

Our customers often trigger a CSAT survey using our incoming webhooks and workflows when a support case is closed in their CRM system like Salesforce or Zendesk.

We notice that survey-feedback responses often reference a team or specific person that the customer has engaged with. Auto-tagging this feedback as “People” with applicable sentiment will provide these companies with an easy way to measure and track how customers are feeling about the people aspect of a company’s Customer Success or Support program.