I Always Get Sentimental (Analysis) This Time of Year

Here’s one way:compound = []positive = []neutral = []negative = []for x in comments: x_comp = analyzer.polarity_scores(x)["compound"] x_pos = analyzer.polarity_scores(x)["pos"] x_neu = analyzer.polarity_scores(x)["neu"] x_neg = analyzer.polarity_scores(x)["neg"] compound.append(x_comp) positive.append(x_pos) neutral.append(x_neu) negative.append(x_neg) print(f'compound: {compound}, length: {len(compound)}')print(f'positive: {positive}, length: {len(positive)}')print(f'neutral: {neutral}, length: {len(neutral)}')print(f'negative: {negative}, length: {len(negative)}')Let’s look at our aggregate results..When I manually analyzed approximately 400 comments, this was the distribution of sentiments..It takes roughly and hour for a seasoned analyst with Healthcare experience and an understanding of the full data set to perform our manual “comment analysis.”= 4 hours of manpower…and the result of the Python and VaderSentiment code running on these approximately 26,000 words?< 4 seconds of computing powerUhh… this looks VERY different..However, there is a silver lining..The compound score is -0.6951..If we loosely compared that to our manual analysis, the negative ratio is 0.85, we might be getting somewhere..However, I’m not sure where, yet.THE GHOST OF SENTIMENT FUTURESo, what have we learned?PRO: At a macro level, the Python sentiment analysis compound score recognized the negativity of the overall data set..Like I stated in the introduction to Vader, if you need quick and dirty, this is your tool.CON: However, even normalizing all of the assumptions we’ve made up to this point, a difference of 15.49% in that aggregate score is huge..This is especially true, since we’re talking about a data set that considers 1% deltas to be average and 5% to be significant..If I told a CEO that 69.51% of the comments are negative, he or she would not be pleased..If I later came back and told that CEO I double-checked, manually combed the data, and as a result, actually 85% of the comments are negative, I would probably be shown the door..The gap in understanding isn’t small enough, yet.PRO: Sentiment analysis tools can provide quick and easy results that classify blobs of text into positive, neutral, negative, and more.CON: With VaderSentiment anyway, it was difficult to understand the exact meaning of the positive, neutral, and negative breakout values.. More details

Leave a Reply