What you don’t know about Ridge vs Lasso ?
ridge vs lasso, visualized 👀
Have you ever wondered that why does our model performs best on the training data, sometimes even 100% accuracy, and does entirely 💥 worst on our testing data, this scenario is what we call Overfitting 😧.
As we can see, in this case the 💜 purple line performs nicely on test data compared to 💚green squiggly line, even though the green line performs perfectly on the training data.
In order for us to overcome this problem of overfitting, we add some penalty to the model so that it does not fit the training data perfectly.
📑 Regularized Regression Models.
👉Ridge Regression 😃
In Ridge Regression we add regularization term (l2 norm) to our cost function.
We can see that regression line with alpha=10 does not fit the data nice as compared to the blue line, this makes our model less prone to overfitting.
Note: Value of slope will be closer to zero at higher values of alpha but never equals to zero
👉Lasso Regression 😄
In Lasso Regression we add regularization term (l1 norm) to our cost function. This makes the weights of least import features zero.
At higher values of alpha the slope will be equal to zero.
👉Elastic Regression 😲
The Elastic Regression is basically the middle ground between the Lasso and the Ridge we add both regularization term |l1 norm| and |l2 norm| to the cost function
→ r represent how much each regularization term contributes to cost function.
