The Co-efficients of predictors in Lasso

Question

There's this question in Introduction to Statistical Learning that says:

Suppose that $n=2$,$p=2$,$x_{11}=x_{12}$,$x_{21}=x_{22}$. Furthermore, suppose that $y_1+y_2=0$ and $x_{11}+x_{12}=0$ and $x_{12}+x_{22}=0$, so that the estimate for the intercept in a least squares, ridge regressions, or lasso mode is zero,$\hat{\beta_0}=0$.
(d) Argue in this setting, the lasso coefficients $\hat{\beta_1}$ and $\hat{\beta_2}$ are not unique-in other words, there are many possible solutions to the optimization problem in (c). Describe these solutions.

Information: Here (c) refers to the lasso optimization of the given data, which is:
$$(y_1-\hat{\beta_1}x_{11}-\hat{\beta_2}x_{12})^2+(y_2-\hat{\beta_1}x_{21}-\hat{\beta_2}x_{22})^2+\lambda(|\hat{\beta_1}|+|\hat{\beta_2}|)$$

Now, looking at the solution from this site:

The Problem:
Here, the author says:

Now solutions to the original Lasso optimization problem are contours of the function $(y_1-(\hat{\beta_1}+\hat\beta_2)x_{11})^2$ that touch the Lasso-diamond $\hat\beta_1+\hat\beta_2=s$.

Now, the problem with this line is the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$. It has 4 variables, namely $\hat\beta_1$,$\hat\beta_2$,$x_{11}$ and $y_1$. So should this be plotted in 4-dimensions?
No, because in the next line the author says that :

Finally, as $\hat\beta_1$ and $\hat\beta_2$ vary along the line $\hat\beta_1+\hat\beta_2=\frac{y_1}{x_{11}}$, these contours touch the Lasso-diamond edge $\hat\beta_1+\hat\beta_2=s$ at different points.

In which the author explicitly says $\hat\beta_1+\hat\beta_2=\frac{y_1}{x_{11}}$ a line, which means there are only two dimensions. And that may also mean that the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$ might be 3-dimensional, and it's contours are represented in 2-dimensions.

But how do $x_{11}$ and $y_1$ vary in this setting, given that two dimensions are taken by $\hat\beta_1$ and $\hat\beta_2$, (as they are used to make that Lasso-diamond)?

So, basically the representation of the function $(y_1-(\hat\beta_1+\hat\beta_2)x_{11})^2$ is not clear to me.

Answer 1

The function that you say should be plotted in 4 dimensions is in fact the residual sum of squares, defined as $$ f(\hat\beta_1, \hat\beta_2, y_1, x_{11}) = (y_1 - (\hat \beta_1 + \hat\beta_2)x_{11})^2 $$ My intuitive explanation would be to treat $y$s and $x$s as parameters, i.e. fixed for any given optimization problem. Thus, both this function and the penalty term, which does not depend on the data, can be plotted in 3 dimensions (two for the $\hat\beta$s and one for the value of the function), or as contours in 2 dimensions.
(Different values of the data "parameters" would give different shapes of the RSS-contours, such as circles or ellipses instead of a line.)