Hyperparameter

Also: hyperparameters, hyper-parameter

Hyperparameters control the model's structure or the training process: learning rate, batch size, number of layers, dropout rate, regularization strength. They are tuned by comparing validation performance.

Example

Gradient descent learns the weights, but you pick η = 0.001, batch size 64 and dropout 0.5 up front, then try other values and keep whichever scores best on the validation set.

See it in the lesson: Gradient Descent Variants →

Related terms

More in Optimization & Training