Ridge Regression - what is the relationship between $\lambda$ and the data's noise? Currently reviewing some problems in my ML class, and I came across this problem:

You estimate a ridge regression model with some data taken from your robot, and
find (using cross validation) and optimal ridge penalty $\lambda_1$. You then buy a new sensor which has noise with $1/4$ the variance (half the standard deviation) as before. Using the same number of observations as before you collect new data, and find a new optimal ridge penalty $\lambda_2$.
Which of the following will be closest to true?

*

*$\lambda_1 / \lambda_2 = 1/4$

*$\lambda_1 / \lambda_2 = 1/2$

*$\lambda_1 / \lambda_2 = 1$

*$\lambda_1 / \lambda_2 = 2$

*$\lambda_1 / \lambda_2 = 4$

The solution key indicates that the answer is $\lambda_1 / \lambda_2 = 4$, but does not provide an explanation.
Here is my attempt at solving the problem:
From an assumption that $y_i = \mathbf{x}_i^\top \mathbf{w}^* + \epsilon_i$ where each error epsilon is drawn from a gaussian distribution with 0 mean and variance $\sigma^2$ (i.e. $\epsilon_i \sim N(0, \sigma^2)$), we have that ridge regression is
$$
\widehat{\mathbf{w}}_\text{ridge} = \arg\underset{\mathbf{w}}{\min} \sum_{i=1}^{n}(y_i - \mathbf{x}_i^\top  \mathbf{w})^2 + \lambda \|\mathbf{w} \|_2^2
$$
The closed form solution for $\widehat{\mathbf{w}}_\text{ridge}$ is
$$
\widehat{\mathbf{w}}_\text{ridge} = (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\mathbf{X}^\top\mathbf{y}
$$
When calculating the bias-variance tradeoff, we get
$$
\begin{aligned}
\widehat{\mathbf{w}}_\text{ridge} &= (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\mathbf{X}^\top\mathbf{y} \\
&= (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\mathbf{X}^\top(\mathbf{X}\mathbf{w}^* + \mathbf{\epsilon}) && \mathbf{y} = \mathbf{X}\mathbf{w}^* + \mathbf{\epsilon} \\
&= (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}(\mathbf{X}^\top\mathbf{X}\mathbf{w}^* + \mathbf{X}^\top\mathbf{\epsilon}) \\
&= (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}(\mathbf{X}^\top\mathbf{X}\mathbf{w}^* + \mathbf{X}^\top\mathbf{\epsilon} + \lambda\mathbf{I{w^*}} - \lambda\mathbf{I{w^*}}) &&\text{adding zero} \\
&= (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}((\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})\mathbf{w}^* - \lambda\mathbf{I{w^*}} + \mathbf{X}^\top\mathbf{\epsilon}) &&\text{rearranging terms} \\
&= \mathbf{w}^* - (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\lambda\mathbf{w}^* + (\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\mathbf{X}^\top\mathbf{\epsilon} &&\text{distribute}
\end{aligned}
$$
If our sensor has noise with $1/4$ the variance, then our new error ${\epsilon_\text{new}}_i \sim N(0, \frac{\sigma^2}{4}) = \frac{{\epsilon_\text{old}}_i}{2}$. From there, we can conclude that $\mathbf{\epsilon}_\text{new} = \frac{\mathbf{\epsilon}_\text{old}}{2}$.
Does this somehow translate into the variance term $(\mathbf{X}^\top\mathbf{X} + \lambda\mathbf{I})^{-1}\mathbf{X}^\top\mathbf{\epsilon}$? I'm struggling to see how $\lambda_1/\lambda_2 = 4$, as this would imply that $\lambda_\text{new} = \frac{\lambda_1}{4}$. Is there some form of correlation between the distribution that $\epsilon$ is drawn from and $\lambda$?
Would appreciate any suggestions to approach this problem - please let me know if my logic is also off anywhere. Thanks in advance!
 A: The naive approach will be to use the cost function $$\sum_{i=1}^n (\hat{y} - y)^2 + \lambda \sum_{i=1}^p \beta_i^2$$ and consider that when the first term decreases by 4 because of the better sensor, then the second term can be decreased by 4 as well.
However, possibly it is not so trivial. When we decrease the penalty $\lambda$ by a factor $4$, then this will have a sort of feedback effect on the first term which initially decreased by $4$ but now will be larger again because the predictions $\hat{\beta}$ have changed.
What we can say is that the answer will be between two values $1< \lambda_1/\lambda_2 < 4$ but whether it is closer to $1$, $2$, or $4$ depends on the specific problem.
A: Maybe this problem is motivated by a connection to Bayesian Ridge Regression with assumptions $\mathbf y\,|\,\mathbf w, \sigma^2 \sim \mathcal N(\mathbf X \mathbf w, \sigma^2\mathbf I)$ and $\mathbf w \,|\, \tau^2 \sim \mathcal N(\mathbf 0, \tau^2\mathbf I)$. In this setting, maximizing the posterior density $p(\mathbf w\,|\, \mathbf y, \sigma^2, \tau^2)$ w.r.t. $\mathbf w$ for fixed $\sigma^2, \tau^2$ is equivalent to minimizing the Ridge-penalized
least squares criterion for a fixed $\lambda \equiv \sigma^2/\tau^2$ since
$$
\begin{align}
p(\mathbf w\,|\, \mathbf y, \sigma^2, \tau^2)& \propto p(\mathbf y\,|\,\mathbf w, \sigma^2) \cdot p(\mathbf w \,|\, \tau^2)\\
& \propto \exp\left(-\frac{1}{2\sigma^2}\left\|\mathbf y - \mathbf X \mathbf w\right\|_2^2\right) \cdot \exp\left(-\frac{1}{2\tau^2}\left\|\mathbf w\right\|_2^2\right)
\\
&=\exp\left(-\frac{1}{2\sigma^2}\left(\left\|\mathbf y - \mathbf X \mathbf w\right\|_2^2 + \frac{\sigma^2}{\tau^2}\left\|\mathbf w\right\|_2^2\right)\right).
\end{align}
$$
However, as pointed out in Sextus Empiricus' answer, this does not imply that when $\sigma^2$ changes the optimal (w.r.t. cross-validation) $\lambda$ has to change by the same factor.
