# (Linear regression) Can I train and validate at the same time using the following approach?

In a lot of material I found online, training and validation seems to be an iterative process

For example, the regularized regression problem

$$E = \|Xw - t\|_2^2 + \lambda \|w\|^2_2$$

$$X$$ is data matrix, $$w$$ is weights of linear predictors, $$t$$ is targets.

Their algorithm seems to be,

• First, we find $$w^\star$$ by minimizing $$E$$ for some $$\lambda$$,

• Next, use $$w^\star$$ to find the error on validation set

• Then solve $$w^\star$$ again by minimizing $$E$$ for some different $$\lambda$$,

• Next, use $$w^\star$$ to find the error on validation set

$$\vdots$$

• Pick the best performing $$w^\star$$ on the validation set.

What I have in the my code is to compute several weights $$w^\star_k$$ at the same time, and pick the best one based on validation set.

I want to perform training and validation in one shot

• Find $$w^\star_1, \ldots, w_N^\star$$ ($$N$$ different predictors) by minimizing $$E$$ on training set for some $$\lambda_1, \ldots, \lambda_N$$ ($$N$$ different regularization constants),
• Run all of $$w^\star_1, \ldots, w_N^\star$$ on validation set.
• Pick best $$w^\star_k$$, $$k \in \{1, \ldots, N\}$$

Is this a proper way of choosing my hyperparameter?

Sorry if this seems to be a basic question. First time doing "validation"/hyperparam tuning.