You have the low dimension data that you can easily visualize using either matplotlib or seaborn plot.
As a suggestion, you can use the sklearn TSNE module but it is slow on a fairly large dataset.
So there is an upgraded version of TSNE (which is the same as sklearn TSNE module) called Multicore t-SNE which runs the model in parallel and very fast on a large dataset.
Here is a very beautiful blog by Dmitry Ulyanov which clearly explains how to install and use Multicore t-SNE.
As Multicore t-SNE works in a parallel fashion so while defining the model you have to pass the number of CPU cores as a parameter.
# n_jobs is the number of CPU core to run parallelmodel = mTSNE(n_jobs=4, n_components=2)Below is both the visualization of MNIST data using PCA and TSNE.
You can clearly now understand why TSNE is state of the art in visualizing higher dimensional data into a lower dimension.
PCA Visualization on MNIST datasetTSNE Visualization on MNIST datasetFor the whole source code of applying TSNE on MNIST data, you can refer to this beautiful GitHub blog maintained by O'Reilly media.
Drawbacks of TSNEGreat things always come with a cost, so as TSNE.
There are a couple of limitation of TSNECrowding problem is one of the limitations of TSNE, although Student’s T Distribution helped a lot surely, but it doesn’t guarantee you to preserve all the neighborhood points in the lower dimension, it only tries it’s best to come up with the best solution to preserve local structures of the data.
TSNE is computationally expensive than PCA, although I have suggested using multicore t-sne, still, the multicore t-sne is fairly computationally expensive than PCA.
ConclusionAs I have mentioned that TSNE is a very new technique (official paper published in 2008), so you can guess that a very high and advanced mathematical foundation was used to create such a masterpiece.
So in this blog, I tried to bring the concept of TSNE as simple as possible with geometric intuition and how each of the terms (T-Distribution, Stochastic, Neighborhood, Embedding) was used.
I guess it would surely help you to have a better understanding of a layman’s term how TSNE works.
Wish you a happy machine learning :).