Using Natural Language Processing To Rate The Sentiment of the Game of Thrones FinaleUsing Vader and TextBlob to analyze the sentiment of Tweets and reviews by major publicationsFhel DimaanoBlockedUnblockFollowFollowingMay 22As an aspiring Data Scientist, the bootcamp I am attending, Flatiron School, has introduced us to Natural Language Processing.
After learning about tokens, stems, lemmas and corpora, one thing that grabbed my attention was sentiment analysis.
Sentiment analysis, also known as opinion mining, is the automated process of identifying and categorizing a piece of text to determine whether the attitude conveyed is positive, negative or neutral.
Here is an example of sentiment analysis in use.
A travel website aired a commercial advertising their product.
After multiple airings, consumers found the ad extremely irritating and voiced their opinions on social media.
After a few weeks, over half of the social media mentions were negative.
The company recognized this and to remedy the situation, aired a new version of the advertisement where the irritating item, in this case, a violin, was smashed.
After learning about sentiment analysis, my first thought about its use did not have anything to do with travel websites, but of a massive juggernaut of a hit show that so happened to have just aired its very last episode.
I am, of course, talking about Game of Thrones.
The truncated final season only has six episodes and apparently also has a truncated form of storytelling as well.
The internet, however, has not been short on sharing their opinions about the show.
Complaints of poor pacing, characters making uncharacteristically bad decisions and grey morality turning black and white have been some of the loudest complaints.
There is even a change.
org petition to remake the entire season with competent writers that has been signed by more than 1.
4 million people.
Using sentiment analysis, I wondered how the final episode would fare.
I wanted people’s immediate thoughts and reactions .
I scraped twitter for tweets made within twenty four hours after the final episode had aired that contained #GameofThrones or #GameofThronesFinale.
To analyze the sentiment of these tweets, I used VADER and TextBlob.
VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media.
It measures sentiment through positive polarity or negative polarity, and intensity (compound).
TextBlob is Python library for processing textual data and measures sentiment through polarity ranging from -1.
0 to 1.
0, where 1 means a positive statement and -1 means a negative statement.
Another metric TextBlob uses is subjectivity which ranges from 0 to 1.
0, where 0 is very objective and 1.
0 is very subjective.
Negative PolarityHere is an example of a tweet that scored high in the negative polarity with a score of 0.
980 and a very high compound score of -0.
998, meaning it is very negatively intense.
VADER: Pos 0, Neg 0.
98, Neu 0.
02, Compound -0.
9983The following tweet has a high negative polarity of 0.
65 and an intensity score of -0.
7003.
VADER: Pos 0, Neg 0.
659, Neu 0.
341, Compound -0.
7003TextBlob: Polarity -0.
375, Subjectivity 0.
5The following tweet had a high negative polarity, but in my opinion is a positive, even empowering, statement.
With only four words and without more context, NLP gives it a negative polarity which may not accurately convey the attitude of the tweet.
VADER: Pos 0, Neg 0.
728, Neu 0.
272, Compound -0.
8415TextBlob: Polarity -0.
7, Subjectivity 0.
666667Positive PolarityThe following tweet has a positive polarity of 0.
552 and TextBloB found that it had a polarity of 1, so it was as positive in sentiment as it could get.
Based on the context of just text, I would not be able to differentiate if the statement is sincere or sarcasm.
My default assumption, like VADER, would be that this is a positive statement.
If this was spoken, one would be able to listen for other context clues, such as the intonation and stress of the words.
In this example, the true attitude can only be interpreted after analyzing the picture attached with the tweet.
It is sarcasm, it poses a positive statement as a negative sentiment.
VADER: Pos 0.
552, Neg 0, Neu 0.
448, Compound 0.
7125TextBlob: Polarity 1, Subjectivity 0.
3The following tweet has a positive polarity of 0.
587 and a TextBlob polarity of 0.
6.
On its own, the phrase “I love you” returns a positive polarity of 0.
808.
VADER: Pos 0.
587, Neg 0, Neu 0.
413, Compound 0.
7964TextBlob: Polarity 0.
6, Subjectivity 0.
6The next tweet has a positive polarity of 0.
552 and a TextBlob polarity of 1.
VADER: Pos 0.
552, Neg 0, Neu 0.
448, Compound 0.
7125TextBlob: Polarity 1, Subjectivity 0.
3Neutral PolarityThe following tweet was also found to be neutral by VADER but TextBlob finds that it had a negative polarity of -0.
4.
I wonder if it’s the combination of “Game of thrones season 8” that gives it a negative polarity.
Another tweet that had a neutral polarity:VADER: Pos 0, Neg 0, Neu 1, Compound 0TextBlob: Polarity -0.
4, Subjectivity 0.
4So what was the general sentiment of the episode?After over 1800 tweets, VADER and TextBlob returned the following results:The results were pretty surprising.
I was expecting a much higher negative sentiment towards the final episode, instead, the mean sentiment was more positive than negative.
Have I been listening to the very vocal and passionate minority?.Was the episode not that bad or were there other factors at play here?Limitations of Natural Language Processing on Tweets.
The problem with only using NLP to determine sentiment is how meme heavy Twitter is.
The full context or sentiment of a tweet cannot be determined with text alone.
I would need to add optical character recognition and image recognition to determine a more accurate sentiment.
A hard cap of 280 characters can be very light on context clues as well.
Styles of speech such as sarcasm, is very difficult to assess with text alone.
Sarcasm is all about the context and tone of voice, which is why it works better verbally.
How did the finale fare with critics?I wanted to compare the level of sentiment of tweets and reviews of the finale by major publications.
This method was not limited to 240 characters and was not so dependent on images or memes to convey sentiment.
I was curious to see any differences.
How was intensity?.Is there a writing style that correlates to being classified as “more professional”?.Would the compound score be more neutral?For example, a tweet with the highest compound (intensity) score was a F bomb laden tirade about Jon Snow betraying DaenarysI analyzed full reviews of the Washington Post, New York Times, and the BBC.
I also analyzed the Metacritic blurbs of NYTs (mixed review) and BBC (positive review with a score of 80) to see if there was a difference and to possibly gain some insight into Metacritic’s scoring system.
I was very surprised (again) to see that the reviews returned a higher positive sentiment.
My assumption on a more neutral compound score was completely wrong.
The compound scores were more extreme with positive sentiment.
It was interesting to see that BBC’s review with a positive metacritic score of 80 didn’t not score as high in the positive sentiment as NYT’s mixed metacritic review.
VADER: Pos 0.
315, Neg 0, Neu 0.
685, Compound 0.
9626TextBlob: Polarity 0.
263889, Subjectivity 0.
541667As unhappy as I was with the last season (pacing was awful, truncated episode length was a disservice to fans, character changes with no time to marinate), ultimately, Game of Thrones was a captivating thrill ride with great drama, interesting character arcs and beautiful cinematography (and dragons!!).
It had kept me on the edge of my seat and under its thrall since they had the actual gall to execute Ned Stark.
I thoroughly enjoyed the journey, even if the end did not meet its full potential.
‘pos’: 0.
175, ‘neg’: 0.
081, ‘neu’: 0.
745, ‘compound’: 0.
8854, TBPolarity: 0.
1111Resources:How to scrape Historical Twitter Data using Web Scraper and ChromeHere is a step by step video to scrape historical tweets…www.
scrapehero.
comcjhutto/vaderSentimentVADER Sentiment Analysis.
VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment…github.
comWhy the Writing in Season 8 of 'Game of Thrones' Feels OffWith each new episode of Game of Thrones , complaints about the hurried and inconsistent writing seem to mount…www.
wired.
comTextBlob: Simplified Text Processing – TextBlob 0.
15.
2 documentationfrom textblob import TextBlob text = ''' The titular threat of The Blob has always struck me as the ultimate movie…textblob.
readthedocs.
iEpisode Review: Game of Thrones Series FinaleUpdated May 21 Spoiler warning: This page may contain descriptions of events in this and previous episodes.
Watch…www.
metacritic.
com.