First, move into the src folder by using the chdir() just like we did before: View the code on Gist.
Then, import the required libraries: View the code on Gist.
Note: model, sample and encoder are the Python files present in the src subfolder of the main GPT-2 folder: View the code on Gist.
Let’s understand the parameters we just saw one-by-one: model_name: This indicates which model we are using.
In our case, we are using the GPT-2 model with 345 million parameters or weights seed: Integer seed for random number generators, fix seed to reproduce results nsamples: This represents the number of sample texts generated in our output batch_size: This only affects speed/memory.
This must also divide nsamples Note: To generate more than one sample, you need to change the values of both nsamples and batch_size and also have to keep them equal.
length: It represents the number of tokens in the generated text.
If the length is None, then the number of tokens is decided by model hyperparameters temperature: This controls randomness in Boltzmann distribution.
Lower temperature results in less random completions.
As the temperature approaches zero, the model will become deterministic and repetitive.
Higher temperature results in more random completions top_k: This parameter controls diversity.
If the value of top_k is set to 1, this means that only 1 word is considered for each step (token).
If top_k is set to 40, that means 40 words are considered at each step.
0 (default) is a special setting meaning no restrictions.
top_k = 40 generally is a good value models_dir: It represents the path to parent folder containing model subfolders (contains the <model_name> folder) Now, the time has come to witness the result produced by the most advanced language model.
Let’s run this function and generate some text (be prepared to be stunned): View the code on Gist.
You will now be asked to enter a string.
This is what I went with: I went to a lounge to celebrate my birthday and And here’s what my GPT-2 text generator came up with: I called Donna and told her I had just adopted her.
She thought my disclosure was a donation, but I’m not sure if Donna met the criteria.
Donna was a genuinely sweet, talented woman who put her life here as a love story.
I know she thanked me because I saw her from the photo gallery and she appreciated my outrage.
It was most definitely not a gift.
I appreciate that I was letting her care about kids, and that she saw something in me.
I also didn’t have much choice but to let her know about her new engagement, although this doesn’t mean I wasn’t concerned, I am extremely thankful for all that she’s done to this country.
When I saw it, I said, “Why haven’t you become like Betty or Linda?” “It’s our country’s baby and I can’t take this decision lightly.
” “But don’t tell me you’re too impatient.
” Donna wept and hugged me.
She never expresses milk, otherwise I’d think sorry for her but sometimes they immediately see how much it’s meant to her.
She apologized publicly and raised flagrant error of judgment in front of the society of hard choices to act which is appalling and didn’t grant my request for a birth certificate.
Donna was highly emotional.
I forgot that she is a scout.
She literally didn’t do anything and she basically was her own surrogate owner.
August 11, 2017 at 12:11 PM Anonymous said… Incredible!.I was speechless the first time I saw this result.
The incredible level of detail and the impressive grammar – it’s almost impossible to tell that it was generated completely by a machine.
Pretty impressive, right?.Go ahead and play around with the input string and share your results in the comments section below.
A Note on the Potential Misuse of GPT-2 GPT-2 has been in the news for its possible malicious use.
You can imagine how powerful this NLP framework is.
It could easily be used to generate fake news or frankly any fake text without humans being able to realize the difference.
Keeping these things in mind, OpenAI didn’t release the full model.
Instead, they have released a much smaller model.
The original model is trained on 40 GB of internet data and has 1.
5 billion parameters.
The two sample models OpenAI have released have 117 million and 345 million parameters.
End Notes In this article, we’ve used the medium-sized model with 345M million parameters.
If these smaller models are capable of generating such impressive results, imagine what the complete model of 1.
5 billion parameters could generate.
Scary and exciting at the same time.
What’s next for NLP?.I have a feeling we won’t have to wait too long to find out.
In the meantime, try out this GPT-2 framework and let me know your experience below.
Besides that, I also encourage you to use this model only for the purpose of research and to gain knowledge.
Keep learning!.You can also read this article on Analytics Vidhyas Android APP Share this:Click to share on LinkedIn (Opens in new window)Click to share on Facebook (Opens in new window)Click to share on Twitter (Opens in new window)Click to share on Pocket (Opens in new window)Click to share on Reddit (Opens in new window) Related Articles (adsbygoogle = window.
adsbygoogle || ).