There's this question in Introduction to Statistical Learning that says:
Suppose that $n=2$,$p=2$,$x_{11}=x_{12}$,$x_{21}=x_{22}$. Furthermore, suppose that $y_1+y_2=0$ and $x_{11}+x_{12}=0$ and $x_{12}+x_{22}=0$, so that the estimate for the intercept in a least squares, ridge regressions, or lasso mode is zero,$\hat{\beta_0}=0$.
(d) Argue in this setting, the lasso coefficients $\hat{\beta_1}$ and $\hat{\beta_2}$ are not unique-in other words, there are many possible solutions to the optimization problem in (c). Describe these solutions.
Information:
Here (c) refers to the lasso optimization of the given data, which is:
$$(y_1-\hat{\beta_1}x_{11}-\hat{\beta_2}x_{12})^2+(y_2-\hat{\beta_1}x_{21}-\hat{\beta_2}x_{22})^2+\lambda(|\hat{\beta_1}|+|\hat{\beta_2}|)$$
Now, looking at the solution from this site:
The Problem:
Here, the author says:
Now solutions to the original Lasso optimization problem are contours of the function $(y_1-(\hat{\beta_1}+\hat\beta_2)x_{11})^2$ that touch the Lasso-diamond $\hat\beta_1+\hat\beta_2=s$.
Now, the problem with this line is the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$. It has 4 variables, namely $\hat\beta_1$,$\hat\beta_2$,$x_{11}$ and $y_1$. So should this be plotted in 4-dimensions?
No, because in the next line the author says that :
Finally, as $\hat\beta_1$ and $\hat\beta_2$ vary along the line $\hat\beta_1+\hat\beta_2=\frac{y_1}{x_{11}}$, these contours touch the Lasso-diamond edge $\hat\beta_1+\hat\beta_2=s$ at different points.
In which the author explicitly says $\hat\beta_1+\hat\beta_2=\frac{y_1}{x_{11}}$ a line, which means there are only two dimensions. And that may also mean that the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$ might be 3-dimensional, and it's contours are represented in 2-dimensions.
But how do $x_{11}$ and $y_1$ vary in this setting, given that two dimensions are taken by $\hat\beta_1$ and $\hat\beta_2$, (as they are used to make that Lasso-diamond)?
So, basically the representation of the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$ is not clear to me.