Package website: release | dev
mlr3tuning is the hyperparameter optimization package of the mlr3 ecosystem. It features highly configurable search spaces via the paradox package and finds optimal hyperparameter configurations for any mlr3 learner. mlr3tuning works with several optimization algorithms e.g. Random Search, Iterated Racing, Bayesian Optimization (in mlr3mbo) and Hyperband (in mlr3hyperband). Moreover, it can automatically optimize learners and estimate the performance of optimized models with nested resampling. The package is built on the optimization framework bbotk.
mlr3tuning is extended by the following packages.
There are several sections about hyperparameter optimization in the mlr3book.
The gallery features a collection of case studies and demos about optimization.
The cheatsheet summarizes the most important functions of mlr3tuning.
Install the last release from CRAN:
install.packages("mlr3tuning")
Install the development version from GitHub:
::install_github("mlr-org/mlr3tuning") remotes
We optimize the cost
and gamma
hyperparameters of a support vector machine on the Sonar
data set.
library("mlr3learners")
library("mlr3tuning")
= lrn("classif.svm",
learner cost = to_tune(1e-5, 1e5, logscale = TRUE),
gamma = to_tune(1e-5, 1e5, logscale = TRUE),
kernel = "radial",
type = "C-classification"
)
We construct a tuning instance with the ti()
function.
The tuning instance describes the tuning problem.
= ti(
instance task = tsk("sonar"),
learner = learner,
resampling = rsmp("cv", folds = 3),
measures = msr("classif.ce"),
terminator = trm("none")
) instance
## <TuningInstanceBatchSingleCrit>
## * State: Not optimized
## * Objective: <ObjectiveTuningBatch:classif.svm_on_sonar>
## * Search Space:
## id class lower upper nlevels
## 1: cost ParamDbl -11.51293 11.51293 Inf
## 2: gamma ParamDbl -11.51293 11.51293 Inf
## * Terminator: <TerminatorNone>
We select a simple grid search as the optimization algorithm.
= tnr("grid_search", resolution = 5)
tuner tuner
## <TunerBatchGridSearch>: Grid Search
## * Parameters: batch_size=1, resolution=5
## * Parameter classes: ParamLgl, ParamInt, ParamDbl, ParamFct
## * Properties: dependencies, single-crit, multi-crit
## * Packages: mlr3tuning, bbotk
To start the tuning, we simply pass the tuning instance to the tuner.
$optimize(instance) tuner
## cost gamma learner_param_vals x_domain classif.ce
## 1: 5.756463 -5.756463 <list[4]> <list[2]> 0.2063492
The tuner returns the best hyperparameter configuration and the corresponding measured performance.
The archive contains all evaluated hyperparameter configurations.
as.data.table(instance$archive)[, .(cost, gamma, classif.ce, batch_nr, resample_result)]
## cost gamma classif.ce batch_nr resample_result
## 1: 5.756463 0.000000 0.4661146 1 <ResampleResult>
## 2: 11.512925 0.000000 0.4661146 2 <ResampleResult>
## 3: -11.512925 5.756463 0.4661146 3 <ResampleResult>
## 4: -11.512925 0.000000 0.4661146 4 <ResampleResult>
## 5: -5.756463 11.512925 0.4661146 5 <ResampleResult>
## ---
## 21: 11.512925 5.756463 0.4661146 21 <ResampleResult>
## 22: -11.512925 11.512925 0.4661146 22 <ResampleResult>
## 23: 0.000000 5.756463 0.4661146 23 <ResampleResult>
## 24: 5.756463 11.512925 0.4661146 24 <ResampleResult>
## 25: -11.512925 -5.756463 0.4661146 25 <ResampleResult>
The mlr3viz package visualizes tuning results.
library(mlr3viz)
autoplot(instance, type = "surface")
We fit a final model with optimized hyperparameters to make predictions on new data.
$param_set$values = instance$result_learner_param_vals
learner$train(tsk("sonar")) learner