Using Sentiment Analysis Without a PhD in Data ScienceHow techniques like sentiment analysis can be used by mere mortalsSteve ChongBlockedUnblockFollowFollowingFeb 15Much has been written about how trends such as artificial intelligence, machine learning, big data, blockchain — pick your buzzword du jour — are transforming our culture, rewiring our neurons, and disrupting industry.
I don’t doubt that there are some pretty fundamental changes happening.
Most of these changes, one would suspect, are driven by corporations that employ armies of data scientists and that have R&D budgets rivaling the GDP of some developing countries.
So, as the COO of a company firmly in the SMB market and who is not a trained data scientist, these trends don’t really apply to me…or do they?I work for a small software company called Projector PSA that builds Professional Services Automation software targeted at helping consulting companies run their businesses.
We compete against rivals that are larger than we are by several orders of magnitude.
Because we can’t blindly throw money at problems, we pride ourselves in finding smarter, more efficient ways of doing things.
In the past, I’ve written or talked about how we’ve used machine learning techniques to fuel lead generation, predictive analytics to improve revenue projections, and neural networks to identify effective project managers.
Today, I thought I’d focus on how we’ve used a combination of sentiment and context analyses to understand product positioning.
I’ll talk a little bit about what sentiment and context analyses are, how we implemented them, what we did with the data, and what conclusions we were able to draw.
The important point here, though, is that this proof of concept was done as a bit of a lark — an experiment to explore the potential of these concepts, rather than a focused investment of millions of dollars or thousands of person hours.
Sentiment AnalysisSo, what is sentiment analysis, anyway, and how can it be useful?There is a huge treasure trove of data out there on blogs, review sites, social media, emails, chat sessions, web pages, and the like.
For our purposes, we targeted all the publicly available reviews and comments that we could find about us and our direct competitors (and each of our respective products).
The problem is that all this information is messy, non-structured, qualitative — you know, the way real humans think.
I think of sentiment analysis as the way to make sense of all this qualitative, written information the same way statistics is used to make sense of quantitative, numerical, structured data.
At its heart, sentiment analysis simply takes a phrase and attempts to determine whether the author was trying to express a positive or negative viewpoint.
For bonus points, some algorithms try to figure out how strongly that perspective is being expressed.
By analyzing the text, counting the number of words known to be positive or negative, and attributing a weight to those words, these algorithms are able to come up with a score that might range from -1 to +1 that represents the overall sentiment of the phrase.
Simple, right?While the overall approach is pretty straightforward, in practice, a huge amount of work needs to go into creating one of these sentiment analysis mechanisms…which is why we did our analysis using someone else’s algorithm.
Built by researchers at the Georgia Tech and made available as an open source library by MIT, the VADER model is a great approach to analyzing sentiment in the internet age.
It is smart enough to be able to capture the expressiveness of modern, informal text filled with all the slang (meh), emoticons ¯_(ツ)_/¯, acronyms (OMG), repeated punctuation!!!, and CAPITALIZATION that would cause your grammar school English teacher to “Tsk!” in disapproval.
I especially like this example from Jodie Burchell about how VADER understands that this statement is anything but neutral:One other nuance VADER understands is what I call the “but” checker (more on this later).
Examples in the code illustrate this with the following equally rich and colorful example:"Today kinda sux!.But I'll get by, lol" # mixed sentiment example with slang and constrastive [sic] conjunction "but"Context AnalysisOK, so if the sentiment analysis gives us a happy/not happy rating (or more properly, a how happy/how unhappy rating) for each phrase, crunching this data could give us an average sentiment rating for us that we could compare to our competitors.
Vaguely interesting, but not all that useful or actionable.
To help, we coupled the sentiment analysis with some context analysis.
Unlike the generalized, industry standard, open-source sentiment analysis, however, the context analysis that we wanted to do was very customized and targeted for our industry and our needs.
This context analysis process was just an exercise in string parsing and tagging.
For instance, if the phrase contained “ease of use” or “user friendly,” that suggests the context is “usability.
” Or, if the words “reports” or “dashboards” occurred, we tagged those phrases as being about the context of “visibility” into the firm’s data.
Because these words are so specific to the insight we were looking to get out of the reviews and comments, we didn’t see any alternative to building some customized categorization mechanisms.
Fortunately, this is a pretty straightforward process that doesn’t require the enormous amount of research that the sentiment analysis requires.
Neither the context nor the sentiment analysis will be flawless as they are attempts to quantify information that is, by its very nature, qualitative and subjective.
The one caution to this process of building custom context analysis algorithms, however, is to take steps to prevent bias, either conscious or unconscious, from creeping into the system.
We wrote these categorization rules without knowing which product each phrase was commenting on to try to eliminate (or at least minimize) any sort of bias.
I suppose I should also note that neither the context nor the sentiment analysis will be flawless as they are attempts to quantify information that is, by its very nature, qualitative and subjective.
What you’re shooting for here is to make sure that any errors are applied equally across all the dimensions (product, categories, timeframes, etc.
) and don’t lead to any systematic bias.
At any rate, what we ended up with was five broad contexts into which we could categorize each phrase:VisualizationWith the sentiment analysis done and the categorization complete, we put together a quick visualization to help people get a general sense of where sentiment fell along the continuum of negative (red), neutral (grey), or positive (green):Perhaps more importantly, this visualization gives the viewer the ability to mouse over different areas to drill in to and isolate different bits of data.
This allows them, for instance, to see how one product fared in each of the five categories, to see how each of the products compared in a single category, and to drill into the individual comments that were included in the analysis:Sentiment Analysis Results Chord DiagramFor the curious, this is a modified chord diagram rendered in a technology called D3.
js that was championed by Mike Bostock at the New York Times, among others.
It’s a visualization library that we often use to create more interactive and compelling visualizations of complex data, such as our interactive case study around forecasting revenue using predictive analytics.
If you’re especially interested in this visualization technique, feel free to play around with our interactive version of the sentiment analysis chord diagram.
ResultsThis isn’t intended to be a post about our product and our competitors, so I’ll keep this section brief.
But, it’s useful to comment on what the real-world, business implications were of this analysis.
What this process helped us to do was to get an overall picture that allowed us to validate things that we thought we knew about our product and our company (e.
g.
, that our users love working with our support team).
This combination of sentiment and context analyses helped us to validate things we already knew — and also helped us dispel some myths that we thought we “knew.
”In addition, it helped us to dispel some myths about some things that we thought we “knew.
” For instance, we were surprised that we fared as well as we did in the product usability scores.
This is an area that we know we have some work to do.
Drilling down into the comments, what we found was that many of them were similar to sentiments such as, “One challenge has been the user interface…but that is currently being overhauled.
” (Remember the “but” checker mentioned earlier?) The VADER model is smart enough to understand that because we’re moving in the right direction with the user interface, the sentiment is not as negative as it might be at first glance.
So, not only have we been able to get an overall map of how we’re doing in each of these categories as compared to our competitors, we’ve been able to drill into our weak points to help inform our product roadmap.
We’ve been able to arm our sales and marketing teams with both the overall competitive analysis as well as with specific comments that can be used to help us compete.
And, we can even use the analysis to target specific companies who are using our competitors’ products in proactive outreach campaigns capable of targeting their individual pain points.
ConclusionsThe point of this whole discussion is not to get too deep into the models, the analyses, or the specific use cases that happened to be useful for our business.
After all, the likelihood that your business is so similar that you’d want to just replicate what we did is minuscule.
Rather, the point of this is to think about these techniques of artificial intelligence or machine learning or sentiment analysis as accessible, attainable, useful tools no different than Microsoft Excel or Google Analytics.
Far from being confined to the exclusive use of well-funded teams of PhDs, data scientists, elite developers, or other shamans, seers, and oracles, these techniques are — right now — immediately employable by mere mortals like us.
Mere mortals working in startups or SMBs to help solve the tangible business problems we face every day.
These techniques of artificial intelligence or machine learning or sentiment analysis are — right now — immediately employable by mere mortals like us.
All it takes is a little creativity and some intellectual curiosity…combined with the hard work those PhDs, data scientists, and elite developers have done to kindly put these technologies into a form that we can use.
Many thanks for their hard work and, in turn, we pledge that it won’t go to waste!.