Using 3D visualizations to tune hyperparameters in ML models

To perform it we have to divide the data in 3 subsets: a train set (used to train the model), a validation set (to optimize the hyperparameters) and a test set (to check the performance of the model at the end as if we were in production already)..We evaluate the performance of the model using some score that will vary depending on the type of problem we’re trying to solve (regression, classification, clustering…).A pic is always worth 1000 words!For regression, R2 (R-squared) works well and is the one I used..Note that in the plots I used the words test set and test score but what I’m referring to is validation.First: 2D HeatmapsAs a quick approach, we could plot heatmaps with the possible pairs of different parameters to see the areas where the maximum R2 is achieved in the test and the train sets..First, let’s create a 3D scatter plot where the size of the points is proportional to the training time and the color is proportional to the R2 score in the test set.See this in fullscreen mode hereNice, but we can’t still see much..We can say that the bigger max_features and the smaller min_samples_split, the greater the test score is, but it’s difficult to hover on points that are in the middle of the 3D scatter plot.The code to make this is too long for Medium, but you can see it all hereBetter: add sliderWe have 3 independent variables, the 3 parameters we’re trying to optimize..A more intuitive approach is to leave the value of one of them fixed and plot a 3D surface of the test R2 score VS the other two variables. This is closer to the 1D validation curves we’re used to see, and if we add a slider to move through the third variable we can effectively see the same information than in the previous plot, but in a much clearer wayFullscreen mode here.Or if you prefer a 3D scatter plot so that you can see each individual point, here it is.. More details

Leave a Reply