I'm a software engineer learning machine learning, particularly through Andrew Ng's machine learning courses. While studying linear regression with regularization, I've found terms that are confusing:
- Regression with L1 regularization or L2 regularization
- Ridge regression
So my questions:
Is regression with L1 regularization exactly the same as LASSO?
Is regression with L2 regularization exactly the same as Ridge Regression?
How is "LASSO" used in writing? Should it be "LASSO regression"? I've seen usage like "the lasso is more appropriate".
If the answer is "yes" for 1 and 2 above, then why are there different names for these two terms? Does "L1" and "L2" come from computer science / math, and "LASSO" and "Ridge" from stats?
The use of these terms is confusing when I see posts like:
"What is the difference between L1 and L2 regularization?" (quora.com)
"When should I use lasso vs ridge?" (stats.stackexchange.com)