# Statistical interpretation of Ridge regression (or least square)

I do not know any advanced statistics; I only took a intro to probability. So please bare with me. If my question violates anything I will delete it right away.

I want to know how the conventional Ridge regression is statistically interpreted, especially in the following artificial setting.

Assume $x$ is a random variable that models samples in $\mathbb{R}^d$; i.e., $x(\omega) \in \mathbb{R}^d$. Also assume $y$ is a random variable that models target vectors paired with samples; i.e., $(x(\omega),y(\omega))$ is a pair of sample and its target (label). In this setting, assume $\widehat{y}$ denotes the prediction made by RR $$\beta^T x.$$

Then my primary question is how this $\beta$ is modeled? More precisely, what statistical assumption on $\beta$ brings about the normal optimization criterion of Ridge regression $$\min_{\beta} \mathbb{E}[||y-\beta^Tx||^2] + \lambda ||\beta||^2 ?$$

And please recommend me a book where I can study things related to this.

• Section 3.4, and especially 3.4.1 of The Elements of Statistical Learning statweb.stanford.edu/~tibs/ElemStatLearn Commented Jun 25, 2017 at 23:24
• I mimic @MarkL.Stone in his suggestion. I have training in statistics but not math, so I would recommend also section 6.2, specifically section 6.2.1 from An Introduction to Statistical Learning, a more accessible version of The Elements of Statistical Learning, written by the same authors. Commented Jun 25, 2017 at 23:50