Naive-elastic net and elastic net variable selection comparison

Question

The elastic net paper (here) introduced the naive-elastic net and elastic net. The coefficient estimates of naive-elastic net are obtained by solving the problem $$\hat\beta_{naive-enet}=\text{argmin}_\beta \big\lVert y - X\beta\big\rVert^2 + \lambda_1\lVert \beta\rVert_1 + \lambda_2 \lVert \beta\rVert^2_2.$$ The elastic net is a scaled version of the naive-elastic net: $\hat\beta_{enet} = (1+\lambda_2)\hat\beta_{naive-enet}$.

But I'm confused about the prostate data analysis in Section 4 of the paper. According to the definition above, the naive-elastic net and elastic net must select the same variables since the scaling above does not change non-zero coefficients. Hence, in the data analysis, the elastic net should have selected all the variables since the naive-elastic net included all the variables in the model. But the authors reported different selected-variables for these estimators in Table 1. How is this possible? (The picture below is from the paper.)

I know that different parameters give different models. But the elastic net is two-step process: 1. fit a naive-elastic net and 2. rescale the fitted naive-elastic net. If I get a naive-elastic net model with all the variables at the 1st step, I should get the model with the same variables in the 2nd step!

Answer 1

It is possible that they select different variables because the parameters $(\lambda,s)$ are different in the two analyses. The equality $\hat\beta_{enet}=(1+\lambda_2)\hat\beta_{naive-enet}$ is only valid when those parameters are identical. (The parameters $(\lambda,s)$ are different because they are estimated through cross validation).

---

EDIT: adding argument for why the tuning parameters are different

Let $\hat\beta_{enet}(X,\lambda,s)$ and $\hat\beta_{naive}(X,\lambda,s)$ be the elastic net and naive elastic net estimates, respectively, based on tuning parameters $\lambda$ and $s$ and dataset $X$. Let $\hat\lambda_{enet},\hat s_{enet}$ be the chosen elastic net tuning parameters. Let $L$ be the loss function to compare the predicted variable $y$ and the predictions, $X\hat\beta$. Let $X^{(1)}, \dots, X^{(10)}$ be the 10 cross validation datasets. Then we can write

$$ \hat\lambda_{enet}, \hat s_{enet} = \arg \min_{\lambda,s} \sum_{i=1}^{10} L(y,X^{(i)}\hat\beta_{enet}(\lambda,s,X^{(i)}))\\ = \arg \min_{\lambda,s} \sum_{i=1}^{10} L(y,(1+\lambda)X^{(i)}\hat\beta_{naive}(\lambda,s,X^{(i)}))\\ $$

Likewise

$$ \hat\lambda_{naive}, \hat s_{naive} = \arg \min_{\lambda,s} \sum_{i=1}^{10} L(y,X^{(i)}\hat\beta_{naive}(\lambda,s,X^{(i)}))\\ $$

The two formulas are different, and so it is very likely that $\hat\lambda_{enet}\neq \hat\lambda_{naive}$ and $\hat s_{enet}\neq \hat s_{naive}$