We write for ESOMAR’s Research World Blog on making sense of nonsensical text data

Most companies have a huge amount of untapped text data at their fingertips, but many are at a loss when it comes to accessing and using it for business decisions. It can seem overwhelming, disjointed and, sometimes, nonsensical. However, our Content Marketing Manager, Alister Houghton, writes on ESOMAR’s Research World blog that, by using text analytics, companies can utilize even the “messiest” language to find greater success and gain a competitive edge.

How does he know? Because we used the Relative Insight platform to analyze the messiest, most confusing data we could find: language describing drug trips. To get the words we needed for text analytics, we conducted a survey of 4,000 people, screening to ask whether or not they had ever tripped using LSD or psilocybin mushrooms. We collected some basic demographic data, and presented an open-end question which “simply asked respondents to describe their trip experience in as much detail as possible.”

As you can imagine, the language was, well, out of this world. While before AI-assisted text analysis, it would have been impossible to make sense of trip reports like these, Alister writes that “using our comparative text analysis platform, which leverages natural language processing, we could take this enormous data set of five million words and surface the key differences between multiple data sets – resulting in a gold mine of information after just a few seconds of processing.”

In the article, he dives into just a few of the insights that surfaced by using comparative analysis, such as the different types of language used between men and women and how different generations discussed their drug trips, and reasons for using drugs in the first place. 

He concludes the piece with: “So if you think making sense of your text data is an insurmountable task, this study helps prove that even the biggest data set with the most fantastical language can be processed, analyzed and help you make critical decisions. Using the right technology, and the power of natural language processing and comparative text analytics, you can make sense of even the most nonsensical text data.”

You can read the complete article here.