How to approach this Lasso problem on the credit dataset I am using the credit dataset to predict the balance of an individual. The specification of the model is non-linear however, in the following form:
$$y_i = \beta_0 + \sum^J_{j=1}\beta_jx_{ij} + \sum^J_{j=1}\sum^K_{k=1}\gamma_{jk}x_{ij}x_{ik} + e_i$$
My understanding of this problem so far is that I am generating new predictors which are combinations of all the original regressors.
Prior to doing any modeling, I have one-hot encoded the dummy variables {cards, gender, student, married, ethnicity} into their own columns to make the data wider, which produces 18 predictors. After the non-linear transformation, I am left with a data frame with size (400 * 189) as the combinations of all predictors generates 171 combinations in addition to the 18 original predictors.
My question is now in relation to whether the dependent variable (balance) is needed to be standardised, this is because with and without standardisation of the dependent variable yields widely varying results.
With Standardisation of Dependent Variable
The model is quite sensitive to choice of penalisation term, for instance alpha = 0.5 reduces all but one coefficient to zero (implying the optimal penalty is going to be somewhere lower than 0.5 anyway). The MSE is easier to interpret and compare as it is between 0 and 1.
Without Standardisation of Dependent Variable
In this case, I see huge increases in the MSE as the penalty increases, where MSE is reported into the thousands - I'm not too sure on what is really going on there, as it makes comparison a bit challenging.
The answer seems somewhat obvious but I am still unsure - would standardisation of the depdendent be the correct approach I should follow?
 A: Changing the mean of the dependent variable makes no practical difference, as all that happens is that $\beta_0$ changes to match that change and you are not penalising $\beta_0$
Rescaling also makes little difference beyond rescaling other parts of the calculation, since trying to minimise
$$\sum^I_{i=1}\left(y_i - \left(\beta_0 + \sum^J_{j=1}\beta_jx_{ij} + \sum^J_{j=1}\sum^K_{k=1}\gamma_{jk}x_{ij}x_{ik}\right)\right)^2 \\- \alpha\left(\sum^J_{j=1}|\beta_j|+\sum^J_{j=1}\sum^K_{k=1}|\gamma_{jk}|\right)$$
or similarly minimise the first term subject to $\sum\limits^J_{j=1}|\beta_j|+\sum\limits^J_{j=1}\sum\limits^K_{k=1}|\gamma_{jk}| \le \alpha$,
then this will produce similar results to trying to minimise
$$\sum^I_{i=1}\left(ky_i - \left(k\beta_0 + \sum^J_{j=1}k\beta_jx_{ij} + \sum^J_{j=1}\sum^K_{k=1}k\gamma_{jk}x_{ij}x_{ik}\right)\right)^2 \\- k\alpha \left(\sum^J_{j=1}|k\beta_j|+\sum^J_{j=1}\sum^K_{k=1}|k\gamma_{jk}|\right)$$
or similarly minimise the first term subject to $\sum\limits^J_{j=1}|k\beta_j|+\sum\limits^J_{j=1}\sum\limits^K_{k=1}|k\gamma_{jk}| \le k\alpha$,
except that in the second case the regularisation parameter needs to be multiplied by $k$, all the optimal coefficients are multiplied by $k$, and the mean-square error is multiplied by $k^2$.
So, with suitable scaling adjustment to the regularisation parameter (which you may be tuning using cross-validation), it makes little or no practical difference whether you standardise the dependent variable or not. (By contrast, standardising the independent variables would usually make a difference if you rescale different independent variables by different amounts.)
