5k views

### Is ridge regression useless in high dimensions ($n \ll p$)? How can OLS fail to overfit?

Consider a good old regression problem with $p$ predictors and sample size $n$. The usual wisdom is that OLS estimator will overfit and will generally be outperformed by the ridge regression estimator:...
5k views

### What are the main theorems in Machine (Deep) Learning?

Al Rahimi has recently given a very provocative talk in NIPS 2017 comparing current Machine Learning to Alchemy. One of his claims is that we need to get back to theoretical developments, to have ...
Consider a simple regression (normality not assumed): $$Y_i = a + b X_i + e_i,$$ where $e_i$ is with mean $0$ and standard deviation $\sigma$. Are the Least Square Estimates of $a$ and $b$ ...