The developers have released a much smaller version of GPT-2 for researchers and engineers to test.
The original model has 1.
5 billion parameters – the open source sample model has 117 million.
Resources to learn and read more about GPT-2: OpenAI’s official blog post Pretrained models for GPT-2 Research paper Word Embeddings Most of the machine learning and deep learning algorithms we use are incapable of working directly with strings and plain text.
These techniques require us to convert text data into numbers before they can perform any task (such as regression or classification).
So in simple terms, word embeddings are the text blocks that are converted into numbers for performing NLP tasks.
A word bmbedding format generally tries to map a word using a dictionary to a vector.
You can get a much more in-depth explanation of word embeddings, it’s different types, and how to use them on a dataset in the below article.
If you are not familiar with the concept, I consider this guide a must-read: An Intuitive Understanding of Word Embeddings: From Count Vectors to Word2Vec In this section, we’ll look at two state-of-the-art word embeddings for NLP.
I have also provided tutorial links so you can get a practical understanding of each topic.
ELMo No, this ELMo isn’t the (admittedly awesome) character from Sesame Street.
But this ELMo, short for Embeddings from Language Models, is pretty useful in the context of building NLP models.
ELMo is a novel way of representing words in vectors and embeddings.
These ELMo word embeddings help us achieve state-of-the-art results on multiple NLP tasks, as shown below: Let’s take a moment to understand how ELMo works.
Recall what we discussed about bidirectional language models earlier.
Taking a cue from this article, “ELMo word vectors are computed on top of a two-layer bidirectional language model (biLM).
This biLM model has two layers stacked together.
Each layer has 2 passes — forward pass and backward pass: ELMo word representations consider the full input sentence for calculating the word embeddings.
So, the term “read” would have different ELMo vectors under different context.
A far cry from the older word embeddings when the same vector would be assigned to the word “read” regardless of the context in which it was used.
Resources to learn and read more about ELMo: Step-by-Step NLP Guide to Learn ELMo for Extracting Features from Text GitHub repository for pretrained models Research Paper Flair Flair is not exactly a word embedding, but a combination of word embeddings.
We can call Flair more of a NLP library that combines embeddings such as GloVe, BERT, ELMo, etc.
The good folks at Zalando Research developed and open-sourced Flair.
The team has released several pretrained models for the below NLP tasks: Name-Entity Recognition (NER) Parts-of-Speech Tagging (PoS) Text Classification Training Custom Models Not convinced yet?.Well, this comparison table will get you there: ‘Flair Embedding’ is the signature embedding that comes packaged within the Flair library.
It is powered by contextual string embeddings.
You should go through this article to understand the core components that power Flair.
What I especially like about Flair is that it supports multiple languages.
So many NLP releases are stuck doing English tasks.
We need to expand beyond this if NLP is to gain traction globally!. Resources to learn and read more about Flair: Introduction to Flair for NLP: A Simple yet Powerful State-of-the-Art NLP Library Pretrained models for Flair Other Pretrained Models StanfordNLP Speaking of expanding NLP beyond the English language, here’s a library that is already setting benchmarks.
The authors claim that StanfordNLP supports over 53 languages – that certainly got our attention!.Our team was among the first to work with the library and publish the results on a real-world dataset.
We played around with it and found that StanfordNLP truly does open up a lot of possibilities of applying NLP techniques on non-English languages.
like Hindi, Chinese and Japanese.
StanfordNLP is a collection of pretrained state-of-the-art NLP models.
These models aren’t just lab tested – they were used by the authors in the CoNLL 2017 and 2018 competitions.
All the pretrained NLP models packaged in StanfordNLP are built on PyTorch and can be trained and evaluated on your own annotated data.
The two key reasons we feel you should consider StanfordNLP are: Full neural network pipeline for performing text analytics, including: Tokenization Multi-word token (MWT) expansion Lemmatization Parts-of-speech (POS) and morphological feature tagging Dependency Parsing A stable officially maintained Python interface to CoreNLP Resources to learn and read more about StanfordNLP: Introduction to StanfordNLP: An Incredible State-of-the-Art NLP Library for 53 Languages (with Python code) Pretrained models for StanfordNLP End Notes This is by no means an exhaustive list of pretrained NLP models.
There are a lot more available and you can check out a few of them on this site.
Here are a couple of useful resources for learning NLP: Natural Language Processing using Python course Certified Program: NLP for Beginners Collection of articles on Natural Language Processing (NLP) I would love to hear your thoughts on this list.
Have you used any of these pretrained models before?.Or you have perhaps explored other options?.Let me know in the comments section below – I will be happy to check them out and add them to this list.
You can also read this article on Analytics Vidhyas Android APP Share this:Click to share on LinkedIn (Opens in new window)Click to share on Facebook (Opens in new window)Click to share on Twitter (Opens in new window)Click to share on Pocket (Opens in new window)Click to share on Reddit (Opens in new window) Related Articles (adsbygoogle = window.
adsbygoogle || []).
push({});.. More details