28 lines
1.1 KiB
Markdown
28 lines
1.1 KiB
Markdown
|
Parameters:
|
||
|
|
variables that are learnt by the model through training.
|
||
|
|
|
||
|
|
|
||
|
|
HyperParameters:
|
||
|
|
variables that are empirical and have to be assigned manually.
|
||
|
|
|
||
|
|
|
||
|
|
Protocol:
|
||
|
|
Train,Test,Validation/Dev Set
|
||
|
|
update HyperParameters and try training with the devset accuracy.
|
||
|
|
pick the best params.
|
||
|
|
Depending on the datasize and the nature of the problem(no of classes to be classified to) decide the test datasize
|
||
|
|
|
||
|
|
Error rate : (Bayes Error rate) lower possible error rate for any classifier of a random outcome.
|
||
|
|
(accuracy of the model shouldn't be more than this,
|
||
|
|
|
||
|
|
Regularization:
|
||
|
|
if it is it means the model is overfitting to the training datas)
|
||
|
|
if the model is overfitting, use regularization to control it.
|
||
|
|
It is a technique to limit the expressiveness of the model.
|
||
|
|
eg.
|
||
|
|
1. L2 regularizer -> Loss' = Loss + lambda*Sum(wi^2) // lambda is the regularization param
|
||
|
|
makes |wi| =~= 0.
|
||
|
|
controls the degree of non-linearity of the model, without having to redesign the model
|
||
|
|
2. Dropout regularizer -> switching off some neurons
|
||
|
|
forcing the model learn from other features(neurons)
|