However, the auctioneers don’t want people selling some random stuff and only want genuine articles so they’ve gone ahead and hired a detective that will first verify all items presented at the auction.
As luck would have it, the detective has his own samples of the original paintings by the famous artist and when you present your random painting, he knows at once that this is nothing like the originals.
Source: GitHubHe rejects it and you decide to give it another try.
But this time, you have a few useful tips, that the detective let slip when he evaluated your canvas, on what the painting should actually look like.
Now when you try your luck again, the painting should be a little better.
But the detective still isn’t convinced and rejects you again.
So you keep trying again and again, every time using some form of feedback to alter the painting and it gets better and better.
(We’re going to assume the detective is OK with you returning endless times.
) In the end, after a thousand or so tries, you’re finally able to come up with something that’s close to a perfect replica.
As the detective looks at his sample paintings, he is unsure whether what you handed him might be one of them or even something else that has the same style and strokes of the famous artist.
What is the step by step process of a GAN’s working?Applying the same thought process to a combination of neural networks, the training of GANs consists of the following steps:Framework of a basic GAN (Source : Medium)The generator initially takes in some random noise and passes it to the discriminator.
As the discriminator already has access to a dataset of real images, it compares them to the image it received from the generator and evaluates its authenticity.
Since the initial image is just random noise it would be evaluated as fake.
The generator keeps trying its luck by varying its parameters so as to produce images that start getting a bit better.
Both networks keep getting smarter as the training progresses, the generator at generating fake images and the discriminator at detecting them.
Eventually the generator manages to create an image indistinguishable from one in the dataset of real images.
The discriminator is not smart enough to tell whether the given image is real or a fake.
At this point, the training ends and the generated image is our final result.
Our own GAN that generates images of Car logosTime to look at some codeHere is a basic Generative network implemented in PyTorch.
Credits : eriklindernorenPros and ConsLike all technologies, GANs also have their own unique set of pros and cons.
Let’s summarize a few of these without going too deep into details.
Here are some potential advantages of using GANs· GANs don’t always need labelled examples to train.
· They are faster at generating samples than other generative models such as belief nets because they have no need to generate the different entries in the sample sequentially.
· They are much easier to train generative models which rely on Monte Carlo approximations to the gradient of the log partition function.
Because Monte Carlo methods don’t work very well in high dimensional spaces, such generative models cannot perform well for realistic tasks like training with ImageNet.
· They don’t introduce any deterministic bias.
Certain generative methods like Variational AutoEncoders introduce deterministic bias because they optimize a lower bound on the log-likelihood rather than the likelihood itself.
This seems to result in VAEs learning to generate blurry samples compared to GANs.
In the same way, there are also the following downsides:· GANs are particularly hard to train.
The function these networks try to optimize is a loss function that essentially has no closed form (unlike standard loss functions like log-loss or squared error).
Thus, optimizing this loss function is very hard and requires a lot of trial-and-error regarding the network structure and training protocol.
· Specifically for Image generation, there is no proper measure to evaluate accuracy.
Since a synthetic image would appear passable to the computer itself, the actual result is a very subjective topic and would depend on a human observer.
Instead we have functions like the Inception Score and Frechet Inception Distance to measure their performance.
Applications of GANsHere comes the fun part.
A list of all the amazing stuff we can do using GANs.
Among all its potential uses, GANs have found a tremendous number of applications in the field of computer vision.
Text to Image ConversionThere are several implementations for this concept such as the TAC-GAN — Text Conditioned Auxiliary Classifier Generative Adversarial Network.
They are used for synthesizing images from their text descriptions.
Left: The structure of a TAC-GAN.
Right: The results produced by feeding a line of text to the network.
Domain TransferGANs have found a great popularity in concepts like style transfer.
Watch the video below:It includes image to image translation using a special type of GAN called CGAN(Conditional Generative Adversarial Networks).
Painting and concept design have never been as easy as this.
However, although GANs can complete doing simple drawings like this purse from its sketch, drawing more complex stuff like perfect human faces is currently not a GAN’s strong point.
In fact, its results are quite nightmarish for certain objects.
Results from the CGAN pix2pix (Source:Github)Image Inpainting and OutpaintingTwo exciting applications of Generative networks can be seen in Inpainting and Outpainting.
The first includes filling up or noise within an image, which might be considered as image repairing.
For example, given an image with holes or gaps, a GAN should be able to correct it in a “passable” fashion.
Outpainting on the other hand involves using the network’s own learning to imagine what an image might look like outside its current boundaries.
Results of Image inpainting(left) and outpainting(right) [Source:Github]Face SynthesisThanks to generative networks, face synthesis is possible, which involves generation of a single face image in different angles.
This is why facial recognition does not require hundreds of samples of your face but can work it out with one.
Not only that, generating artificial faces has also become possible.
NVIDIA recently used their GAN 2.
0 to generate artificial human faces in HD resolution using the Celeba Hq dataset, the first instance of synthetic image generation in High resolution.
Generated faces of imaginary celebrities by a Progressive GAN.
(Image Credits : NVIDIA)GANimationIntricate little methods are also becoming possible such as changing facial movements.
GANimation is a research effort using PyTorch that defines itself as ” Anatomically-aware Facial Animation from a Single Image”.
Official implementation of GANimation.
(Source: Github)Painting to Photograph TranslationAnother example of using GANs to make images more realistic is to simply turn a (pretty good) painting into a photograph.
This is done using a special type of GAN called CycleGAN which uses two generators and two discriminators.
We call one generator G, and have it convert images from the X domain to the Y domain.
The other generator is called F, and converts images from Y to X.
Each generator has a corresponding discriminator, which attempts to tell apart its synthesized images from real ones.
(Source: Github)Where do we go from here ?Machine Learning and GANs are sure to have a tremendous impact on imaging and photography in the near future.
Currently, this technology is capable of generating simple images from text inputs.
However, in the foreseeable future, it would be able to create not only precise images in high resolution but also entire videos.
Imagine an entire movie generated by simply feeding the script into a GAN.
Not only that, every person could use simple interactive apps to create their own movies(possible even starring themselves!).
Would this technology be the end of real photography, direction and acting?Impressive technology also means a potentially usage for wicked purposes.
Perfectly fake images will also require a means to identify and detect them.
Regulation of such image generation would be required.
Currently, GANs are already being used for the creation of fake videos or “Deepfakes” which are being used in a negative manner such as generating fake pornographic videos of celebrities or feature people saying things without their knowledge.
The consequences of making technology to synthesize audio and video generation available to the general population are scary.
Artificial image generation is a double edged technology, especially in a time when little about it is widely known.
Generative Adversarial Networks are a tremendously useful as well as dangerous tool.
The fact that it will reshape the world of technology is certain but how it shall do so, we can only ponder.