For simple linear regression, the regression coefficient is calculable directly from the variance-covariance matrix $C$, by $$ C_{d, e}\over C_{e,e} $$ where $d$ is the dependent variable's index, and $e$ is the explanatory variable's index.

If one only has the covariance matrix, is it possible to calculate the coefficients for a model with multiple explanatory variables?

ETA: For two explanatory variables, it appears that $$\beta_1 = \frac{Cov(y,x_1)var(x_2) - Cov(y,x_2)Cov(x_1,x_2)}{var(x_1)var(x_2) - Cov(x_1,x_2)^2} $$ and analogously for $\beta_2$. I'm not immediately seeing how to extend this to three or more variables.

  • 3
    $\begingroup$ The coefficient vector $\hat{\beta}$ is the solution to $X'Y=(X'X)^{-1}\beta$. Some algebraic manipulation reveals that this is in fact the same as the formula you give in the 2-coefficient case. Laid out nicely here: stat.purdue.edu/~jennings/stat514/stat512notes/topic3.pdf. Not sure if that helps at all. But I'd venture to guess that this is impossible in general based on that formula. $\endgroup$ Commented Jul 22, 2014 at 13:13
  • 1
    $\begingroup$ @David Did you figure out how to extend this to an arbitrary number of explanatory variables (beyond 2)? I need the expression. $\endgroup$
    – Jane Wayne
    Commented May 16, 2016 at 7:47
  • 1
    $\begingroup$ @JaneWayne I'm not sure I understand your question: whuber gave the solution below in matrix form, $C^{-1}(\text{Cov}(X_i, y))^\prime$ $\endgroup$
    – David
    Commented Jun 15, 2016 at 14:19
  • 1
    $\begingroup$ yup I studied it and he's right. $\endgroup$
    – Jane Wayne
    Commented Jun 15, 2016 at 14:52

2 Answers 2


Yes, the covariance matrix of all the variables--explanatory and response--contains the information needed to find all the coefficients, provided an intercept (constant) term is included in the model. (Although the covariances provide no information about the constant term, it can be found from the means of the data.)


Let the data for the explanatory variables be arranged as $n$-dimensional column vectors $x_1, x_2, \ldots, x_p$ with covariance matrix $C_X$ and the response variable be the column vector $y$, considered to be a realization of a random variable $Y$. The ordinary least squares estimates $\hat\beta$ of the coefficients in the model

$$\mathbb{E}(Y) = \alpha + X\beta$$

are obtained by assembling the $p+1$ column vectors $X_0 = (1, 1, \ldots, 1)^\prime, X_1, \ldots, X_p$ into an $n \times p+1$ array $X$ and solving the system of linear equations

$$X^\prime X \hat\beta = X^\prime y.$$

It is equivalent to the system

$$\frac{1}{n}X^\prime X \hat\beta = \frac{1}{n}X^\prime y.$$

Gaussian elimination will solve this system. It proceeds by adjoining the $p+1\times p+1$ matrix $\frac{1}{n}X^\prime X$ and the $p+1$-vector $\frac{1}{n}X^\prime y$ into a $p+1 \times p+2$ array $A$ and row-reducing it.

The first step will inspect $\frac{1}{n}(X^\prime X)_{11} = \frac{1}{n}X_0^\prime X_0 = 1$. Finding this to be nonzero, it proceeds to subtract appropriate multiples of the first row of $A$ from the remaining rows in order to zero out the remaining entries in its first column. These multiples will be $\frac{1}{n}X_0^\prime X_i = \overline X_i$ and the number subtracted from the entry $A_{i+1,j+1} = X_i^\prime X_j$ will equal $\overline X_i \overline X_j$. This is just the formula for the covariance of $X_i$ and $X_j$. Moreover, the number left in the $i+1, p+2$ position equals $\frac{1}{n}X_i^\prime y - \overline{X_i}\overline{y}$, the covariance of $X_i$ with $y$.

Thus, after the first step of Gaussian elimination the system is reduced to solving

$$C_X\hat{\beta} = (\text{Cov}(X_i, y))^\prime$$

and obviously--since all the coefficients are covariances--that solution can be found from the covariance matrix of all the variables.

(When $C_X$ is invertible the solution can be written $C_X^{-1}(\text{Cov}(X_i, y))^\prime$. The formulas given in the question are special cases of this when $p=1$ and $p=2$. Writing out such formulas explicitly will become more and more complex as $p$ grows. Moreover, they are inferior for numerical computation, which is best carried out by solving the system of equations rather than by inverting the matrix $C_X$.)

The constant term will be the difference between the mean of $y$ and the mean values predicted from the estimates, $X\hat{\beta}$.


To illustrate, the following R code creates some data, computes their covariances, and obtains the least squares coefficient estimates solely from that information. It compares them to the estimates obtained from the least-squares estimator lm.

# 1. Generate some data.
n <- 10        # Data set size
p <- 2         # Number of regressors
z <- matrix(rnorm(n*(p+1)), nrow=n, dimnames=list(NULL, paste0("x", 1:(p+1))))
y <- z[, p+1]
x <- z[, -(p+1), drop=FALSE]; 
# 2. Find the OLS coefficients from the covariances only.
a <- cov(x)
b <- cov(x,y)
beta.hat <- solve(a, b)[, 1]  # Coefficients from the covariance matrix
# 2a. Find the intercept from the means and coefficients.
y.bar <- mean(y)
x.bar <- colMeans(x)
intercept <- y.bar - x.bar %*% beta.hat  

The output shows agreement between the two methods:

(rbind(`From covariances` = c(`(Intercept)`=intercept, beta.hat),
       `From data via OLS` = coef(lm(y ~ x))))
                  (Intercept)        x1        x2
From covariances     0.946155 -0.424551 -1.006675
From data via OLS    0.946155 -0.424551 -1.006675
  • 2
    $\begingroup$ Thanks, @whuber! This is exactly what I was looking for, and my atrophied brain was unable to get to. As an aside, the motivation for the question is that for various reasons we essentially do not have the full $X$ available, but have cov(z) from previous calculations. $\endgroup$
    – David
    Commented Jul 22, 2014 at 15:36
  • 13
    $\begingroup$ Answers like this raise the bar of this Cross Validated $\endgroup$
    – jpmuc
    Commented Jul 26, 2014 at 14:12
  • $\begingroup$ @whuber In your example, you computed the intercept from y and x and beta.hat. The y and x are part of the original data. Is it possible to derive the intercept from the covariance matrix and means alone? Could you please provide the notation? $\endgroup$
    – Jane Wayne
    Commented May 16, 2016 at 7:34
  • $\begingroup$ @Jane Given only the means $\bar X$, apply $\hat \beta$ to them: $$\overline X \hat\beta = \overline{X \hat\beta}.$$ I have changed the code to reflect this. $\endgroup$
    – whuber
    Commented May 16, 2016 at 13:50
  • 2
    $\begingroup$ @Migwell On the contrary, Gaussian elimination is a perfectly legitimate mathematical argument, albeit a little unusual in this context. Plenty of proofs--especially in discrete mathematics, combinatorics, optimization, and computational geometry (to name a few areas of mathematics important in statistical applications)--amount to applying "just one method" of solving numerical problems. There is no inherent distinction between "mathematical" and "computational" forms of reasoning. $\endgroup$
    – whuber
    Commented Nov 14, 2021 at 15:51

The answer above is complete and great. The following aims to complement the answer with a different perspective. I also want to bring attention to the fact that using covariances to estimate coefficients you are implicitly assuming there exists a constant factor on you regression, even if you don't calculate it.

Consider the model: $$ Y = 1\alpha + X^T\beta + e $$

Where $1$ is a vector of ones and $X^T$ is a matrix of observable variables. There are $N$ rows.

We can use Frisch-Waugh-Lovell theorem. Consider the projection matrix: $$ M = I - 1(1^T1)^{-1}1^T $$

Then, by FWL, $\beta$ vector can then be retrieved by estimating with the orthogonal components: $$ MY = MX^T \beta + Me $$ $$ (I - 1(1^T1)^{-1}1^T)Y = (I - 1(1^T1)^{-1}1^T)X^T \beta + Me $$ Now you need to realize that $(1^T 1)^{-1} = 1/N$ and $1^T Y$ just sums all values, for example: $$ (1^T 1)^{-1} 1^T Y = \frac{1}{N} \sum_i^N y_i = E(Y) $$ The above expression can be rewritten as:

$$ (Y - 1E(Y)) = (X^T - 1 E(X^T)) \beta + Me $$ That is, this projection matrix $M$ is demeaning $Y$ and $X$. Consider writing the above expression as: $$ \tilde{Y} = \tilde{X}^T \beta + \tilde{e} $$ And we know that OLS estimates are calculated with: $$ \beta = (\tilde{X}'\tilde{X})^{-1} \tilde{X}' \tilde{Y} $$ $$ \beta = [(X^T - 1 E(X^T))' (X^T - 1 E(X^T)]^{-1} (X^T - 1 E(X^T))' (Y - 1E(Y)) $$ $$ \beta = Cov(X)^{-1} Cov(X,Y) $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.