Categorizing Airbnb listing descriptionsFine-tuning BERT on listing descriptions to predict ratingsSteven LiuBlockedUnblockFollowFollowingMar 28An Airbnb home in San Francisco (source)IntroductionOne of the first interactions a guest has with a home is the listing description.
The listing refers to the description of the home, and provides an opportunity to highlight its unique features.
As it turns out, the listing description plays a decisive role in how guests perceive the overall quality of a home.
It is so important that homes qualifying for the Airbnb Plus program (a collection of homes verified for quality, comfort and style) have their listing descriptions written by professional writers trained in the Airbnb Plus editorial style.
But writing a unique and captivating listing description, that stands out from other homes can be difficult.
Airbnb offers resources for writing a great listing description on their blog, and also tips on how to make it more competitive.
“There is nothing to writing.
All you do is sit down at a typewriter and bleed.
”-Ernest Hemingway????Writing can be like this sometimes.
It can be difficult to implement in practice for many hosts, who have no way of receiving feedback until after the guest leaves a rating.
To help hosts learn more about their writing and listing descriptions, I created a text classifier with BERT that enables hosts to receive instant feedback, and see what their predicted rating is.
The dataThe data is available at Inside Airbnb, a site that collects detailed information from some of the major cities Airbnb operates in.
For this project, the data was downloaded from the San Francisco dataset as a series of csv files.
The following visualizations can be found at the accompanying interactive companion site.
Note that it can take a minute to load though, because there is a large amount of data to render.
Play around with it and see what you find!Geospatial analysis of ratings and priceA layered hexbin and point map of ratings and price of Airbnb listings in San Francisco.
The plot highlights a geographically inbalanced supply of homes.
There is a greater demand for homes on the eastern side of the city because it is closer to tourist attractions, downtown and overall commercial activity.
As we move over to West San Francisco, it becomes more residential with fewer attractions in between.
Homes in these areas are set at a lower price as a result.
An exception to this general trend is the Tenderloin, a high-crime neighborhood, which acts to drive down price.
It is also possible that there are a lower supply of homes in West San Francisco due to lifestyle disparities.
In West San Francisco, homes are primarily occupied by families in contrast to young working professionals in East San Francisco.
Families will want to maintain their privacy, and as a result, there are fewer available spaces in West San Francisco.
Related to lifestyle disparities is age gap, which may play a role in the adoption of Airbnb.
As mentioned previously, West San Francisco is a mostly residential neighborhood with an older population.
These individuals may be more reluctant to rent out their homes to strangers, and slow to embrace the idea of home-sharing.
The impact of the inbalanced supply of homes is reflected in the price.
Homes are more expensive in Northeast San Francisco due to the combined influences of the above factors.
The effect of rating on price is most clearly demonstrated in South San Francisco (Ocean View, Portola, Bayview), where lower ratings coincide with low prices.
This effect is mitigated somewhat in the Tenderloin, due to better ratings in the surrounding favorable neighborhoods.
Characteristic terms and associationsScatterplot of terms associated with high star ratings and all other ratings.
One of the more interesting points is how geographical information is captured in the scatterplot.
Neighborhoods like Bernal Heights/Castro are more commonly associated with high star ratings (higher median price), while Cole Valley/Chinatown are associated with all other ratings (lower median price).
The most polar words are private homes (high star rating), and properties owned by home-share companies (all other ratings) such as Sonder & Zeus.
This suggests people may have a better experience with private homes, because it feels more organic.
It is more in line with Airbnb’s vision of living in another person’s home, and immersing yourself in their culture, rather than a company-owned property.
Listing descriptions with high star ratings also tend to feature the home with words such as: victorian, backyard, garden, deck, private entrance and yard.
In contrast, listing descriptions with all other ratings focus more on the amenities of the property with words like: bedding, wifi, linens, screen tv, email, and fitness.
While it is important to list amenities, these same words could also be used to describe a hotel, and does little to describe the individuality of a home.
Topic signals and categoriesA scatterplot of lexical categories associated with high star ratings and all other ratings.
The most frequent topic in listing descriptions with high star ratings is royalty.
This theme is repeated 136 times/25,000 terms versus only 77 times/25,000 terms for all other ratings.
Some of these words include, magnificent, queen, elegant, supreme and distinguished.
However, this is misleading when taken out of context.
The majority of these words are queen/king, and they are used to describe the size of the bed.
Still, these terms are used nearly twice as much, and it can be argued that listing descriptions including more mentions of luxuriously-sized beds, are more likely to attract a higher rating among guests.
The most frequent topic in listing descriptions with all other star ratings is stealing.
Curiously, property appears to be the seed term for this lexical category.
Property is mentioned 108 times/25,000 terms versus 26 times/25,000 terms for high star ratings.
Examining the excerpts show that property is most commonly used in listing descriptions by other home-sharing companies like Sonder & Zeus.
These companies refer to their living spaces as properties, rather than homes.
This is fascinating because it shows that private homeowners don’t view their homes as property, but a home, with all of its associated homey connotations.
On the other hand, companies view these same homes as property, another asset in their portfolio.
When looking at some of the other terms on the list such as negative_emotion, violence and nervousness, keep in mind these categories have been taken out of context.
For example, violence was inferred from:the city’s beating heart, kick back in the suite on its own private floor, hit up The Mill for some liquid energyWord similarityWord similarity clusters are revealed with tSNE.
We observe two semi-distinct shapes, a central cluster (all other ratings) and an outer ring structure (high star ratings).
Geography is clustered in the upper right corner, with neighborhoods and outdoor attractions, like the pier and Alcatraz.
Transportation is captured to the right of the plot, with words like Lyft, Muni, and Bart.
It is interesting to see how similar words are clustered, and then whether these words belong to highly rated descriptions or all others.
For example, one of my favorite word similarities is how closely pet (all other ratings) and kid (high star rating) are clustered together.
Homes more accommodating towards kids, are more favorable, while pets are rarely allowed.
BERTBidirectional Encoder Representations from Transformers (BERT) is a state-of-the-art model released by Google in late 2018.
It is a method for adapting pre-trained language representations for downstream NLP tasks, such as classification.
What makes BERT truly unique is its bidirectionality, meaning, it captures contextual information to the left and right of a word.
Training bidirectional models are inherently difficult because it would allow the word that’s being predicted, to indirectly see itself in a multi-layered context.
This hurdle is overcome by randomly masking some words, and then asking BERT to predict the masked word.
To train BERT to understand relationships between two sentences, during the pre-training process, BERT is further tasked with predicting whether one sentence follows another.
There are two stages in using BERT: pre-training and fine-tuning.
Pre-training is the process of training the BERT model on an extremely large text corpus (Wikipedia) to learn language representations.
This is a highly expensive task (four days of training on 4–16 cloud TPUs!), but thankfully, the researchers at Google have already completed this step and have publicly released their pre-trained models.
All we have to do is download these pre-trained models.
For this project, we will use the Uncased BERT-Base model, which simply means all the text has been lowercased, and accent markers have been stripped.
The BERT-Base model architecture contains 12 Transformer blocks, 768 hidden units, and 12 attention heads.
Fine-tuning is the stage where we adapt the BERT model to our classification task.
On a CPU, this process took 14 hrs, so I recommend using Google’s Colaboratory notebook, and accelerate the process by taking advantage of their cloud GPUs.
WordPiece tokenizationBERT uses WordPiece model tokenization, where each word is further segmented into sub-word units.
A special [CLS] (classifier) token is appended to the beginning of the text, and a [SEP] (separator) token is inserted at the end.
WordPiece allows BERT to capture out-of-vocabulary words and store only 30,522 words, which effectively gives it an evolving word dictionary.
This strategy achieves a good balance between vocabulary size and rare words.
ResultsDepending on your hardware, the training can take a while (14 hrs on a CPU for me!), so if you don’t want to wait around for that long, you should run your code in Colaboratory and take advantage of their GPUs.
{'eval_accuracy': 0.
81263983, 'loss': 0.
57338, 'global_step': 502}The model achieved 81% accuracy on the test set.
Now, let’s try giving the model a listing description to predict.
('We live in an 1895 Victorian top flat with 12 foot ceilings on the best block in San Francisco.
?.Our calendar is always up to date.
Location Location Location~ Our home is close to the Mission,?.Lower Haight, and 2.
5 blocks from the Castro theater.
The underground is 3 blocks away, with a!.street car even closer.
The room is small and cosy, but a great price for one of the most!.expensive neighborhoods in the US!',array([-6.
959856 , -6.
916402 , -6.
4755964, -4.
4082885, -0.
0322696, -4.
1291037], dtype=float32), '3.
0-4.
0')The model returns two things for us.
The first is an array of probabilities for each rating, and the second is the label of the predicted rating.
For this specific chunk of text, the host would expect a 3–4 star rating on their home.
The downside is that each category step is quite wide.
A home receiving 3 stars, and a home receiving 4 stars would both belong in the same category but obviously, there is a significant gap between these two.
Currently, the rating system does not allow me to capture these finer in-between scoring steps, and I recommend obtaining a score of at least 4–5 stars to be confident in your listing description.
TLDRFocus on describing what makes your home unique, instead of amenities.
Does it have a garden for guests to sit out in and enjoy?.Or perhaps a deck with a gorgeous view of the city.These types of descriptions are more apt to earn a high star rating.
Describe your home with terms that evoke feelings of warmth and home, rather than detailing a list of amenities, and describing your space in an impersonal style.
Next stepsCreate a text generator in the Airbnb Plus editorial style, to make it even easier for hosts to quickly create a top-rated listing description.
Train a classifier on a more stratified rating system to help hosts diagnose their listing descriptions in greater detail.
These listing descriptions are unique to San Francisco, and it would be interesting to compare it with listing descriptions from another city, to observe regional discrepancies in what makes a home attractive.
This could also be a fascinating study in how other cultures perceive their homes, as reflected in their text.
Thanks for reading.Stay tuned for more as I continue on my path to become a data scientist.✌️Follow along with the full ???.and code here.. More details