Deep-Learning-Course/ThirdSaturday/Notes.md

Parameters: variables that are learnt by the model through training.

HyperParameters: variables that are empirical and have to be assigned manually.

Protocol: Train,Test,Validation/Dev Set update HyperParameters and try training with the devset accuracy. pick the best params. Depending on the datasize and the nature of the problem(no of classes to be classified to) decide the test datasize

Error rate : (Bayes Error rate) lower possible error rate for any classifier of a random outcome. (accuracy of the model shouldn't be more than this,

Regularization: if it is it means the model is overfitting to the training datas) if the model is overfitting, use regularization to control it. It is a technique to limit the expressiveness of the model. eg.

1. L2 regularizer -> Loss' = Loss + lambda*Sum(wi^2) // lambda is the regularization param makes |wi| =~= 0. controls the degree of non-linearity of the model, without having to redesign the model
2. Dropout regularizer -> switching off some neurons forcing the model learn from other features(neurons)