I have the following data: $n$ observations on $d$ variables $X$ and one outcome variable $Y$; i.e. $X$ is a $n \times d$ matrix and $Y$ an $n \times 1$ vector.

I consider the following Ordinary Least Squares (OLS) model:

$$Y = X \beta + \varepsilon$$

where $\beta$ is a $d \times 1$ vector of coefficients.

I want to scale the data such that the estimated parameter vector has a specific length, i.e. the condition I want to achieve is

$$\Vert\hat{\beta}\Vert^2 := \sum_{i = 1}^{d} \hat\beta_i ^2 = c $$

where $c$ is a constant, say $c = 1$.

More specifically, I normalize the data such that all $d$ covariates have mean = 0 and variance = 1 in the following way:

  1. Define matrix $N$ which is a diagonal matrix with the diagonal elements being $\textrm{diag}(\Sigma_{XX}) )$ where $\Sigma_{XX}$ is the covariance matrix of $X$.
  2. Normalize $X:=X * N^{-1}$

Is it possible to "adjust" this normalizing procedure such that the least-squares parameter vector $\hat\beta := (X^\top X)^{-1} (X^\top Y) $ has a specific norm, namely such that $\sum_{i = 1}^{d} \hat\beta_i ^2 = c$?

By "adjusting" I mean adding factor, which is itself a function of the data $\{X,Y\}$ to matrix $N$ above (or any other way to achieve the desired norm of the estimated parameter vector; I am not wedded to the normalizing approach described in steps 1 and 2 above).


1 Answer 1


I’m not sure what exactly you are looking for(it would be nice to have some more context), but it’s not to hard to scale the data so that $\beta$ has norm 1. The easiest way is to notice that $X\beta=(aX)(\beta/a)$ and then just solve the linear regression once, then rescale all the columns with some scalar $a$. In order to give you a better process for your problem, I’ll need more context as to the reason for having a fixed norm.

  • $\begingroup$ thank you, that was it already! $\endgroup$
    – user194462
    Sep 9, 2020 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.