I do not know any advanced statistics; I only took a intro to probability. So please bare with me. If my question violates anything I will delete it right away.

I want to know how the conventional Ridge regression is statistically interpreted, especially in the following artificial setting.

Assume $x$ is a random variable that models samples in $\mathbb{R}^d$; i.e., $x(\omega) \in \mathbb{R}^d$. Also assume $y$ is a random variable that models target vectors paired with samples; i.e., $(x(\omega),y(\omega))$ is a pair of sample and its target (label). In this setting, assume $\widehat{y}$ denotes the prediction made by RR $$ \beta^T x. $$

Then my primary question is how this $\beta$ is modeled? More precisely, what statistical assumption on $\beta$ brings about the normal optimization criterion of Ridge regression $$ \min_{\beta} \mathbb{E}[||y-\beta^Tx||^2] + \lambda ||\beta||^2 ? $$

And please recommend me a book where I can study things related to this.

  • $\begingroup$ Section 3.4, and especially 3.4.1 of The Elements of Statistical Learning statweb.stanford.edu/~tibs/ElemStatLearn $\endgroup$ Commented Jun 25, 2017 at 23:24
  • $\begingroup$ I mimic @MarkL.Stone in his suggestion. I have training in statistics but not math, so I would recommend also section 6.2, specifically section 6.2.1 from An Introduction to Statistical Learning, a more accessible version of The Elements of Statistical Learning, written by the same authors. $\endgroup$
    – Mark White
    Commented Jun 25, 2017 at 23:50


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.