Hi, There! In this blog, we’ll be discovering various methods towards tuning the hyperparameters of our machine learning model. But wait a minute, What are hyperparameters to the machine learning model? Does it not resemble the gears illustrated in the banner of the blog? Yes, They do! Let me tell you how.
Each of your machine learning models comprises several hyperparameters which upon tweaking and thereby fitting on the processed features from the data eventually steer us through the desired use case. The tweaking of hyperparameter in machine learning models can be considered analogous to the rotation of the gears above!
Now, you know why I put the banner there :P. Nevertheless, Without much beating around the bush, let’s dive into the popular techniques incorporated towards hyperparameter tuning of machine learning models.
You can find the dictionary containing the name of each hyperparameter of a machine learning algorithm as keys through the following command. Later, you can use the keys from the dictionary for hyperparameter tuning over a possible range of acceptable values.
>> KNeighborsClassifier().get_params().keys()Out: dict_keys(['algorithm', 'leaf_size', 'metric', 'metric_params', 'n_jobs', 'n_neighbors', 'p', 'weights'])# similarly other algorithms
>> XGBClassifier().get_params().keys()
Scikit-learn offers GridSearchCV and RandomizedSearchCV for hyperparameter tuning. One can use hyperopt for hyperparameter tuning using Bayesian optimization, SparkML lib or sk-dist built over sklearn for distributed hyperparameter tuning.
GridSearchCV is mostly good when you know a specific range of hyperparameter within which the machine learning algorithms provide the best fit. On the other hand, RandomizedSearchCV is great for discovering new hyperparameters for a machine learning algorithm.
A walkthrough of GridSearchCV and RandomizedSearchCV can be referred in the colab notebook out here. Link1, Link2
Similarly, A walk-through of Bayesian optimization can be referred in the colab notebook out here
One can prefer sk-dist for faster hyperparameter tuning and the same can be explored out here in the blog.
Hope that the blog along with the code walkthrough illustrated in the notebooks will give you enough resources to tune the hyperparameters of your next machine learning algorithm effectively.
With this, I’d like to sign-off!
Until next time!
