Skip to main content
I have corrected formula 1 where the leading $X$ was missing in the definition and the $y$ at the end was too much. See https://en.wikipedia.org/wiki/Projection_matrix#Linear%20model
Source Link

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $h_i$, which is the diagonal of the "hat-matrix", $H$.

In the ordinary least squares case (without linear constraints), this matrix is computed as

$$ H = ( X^T X)^{-1}X^T y \quad \quad \quad (1) $$$$ H = X\left(X^T X\right)^{-1}X^T \quad \quad \quad (1) $$

Where $X$ is the design matrix and $y$ is the variable to be explained. The constrained least-squares regression i'm attempting to do is the following:

$$ \left[ \begin{array}{c} \hat{\beta} \\ \hat{\lambda} \end{array} \right] = \left[ \begin{array}{cc} 2 X^T X & C^T \\ C & 0 \end{array} \right]^{-1} \left[ \begin{array}{c} 2 X^T y \\ d \end{array} \right] $$

where $X$ is again the design matrix, $y$ the variable to be explained and $C$ is the restriction matrix such that

$$ C \beta = d. $$

$\lambda$ is in this case the lagrange multipliers. My Question is then, what is the "hat"-equivalent matrix for this type of regression? Will the formulation in $(1)$ hold? My quess is no, since you are adding additional information to the regression. For instance, if you added 0-restrictions to some of the columns, you might as well have excluded them from the design matrix, in which case the leverages would change.

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $h_i$, which is the diagonal of the "hat-matrix", $H$.

In the ordinary least squares case (without linear constraints), this matrix is computed as

$$ H = ( X^T X)^{-1}X^T y \quad \quad \quad (1) $$

Where $X$ is the design matrix and $y$ is the variable to be explained. The constrained least-squares regression i'm attempting to do is the following:

$$ \left[ \begin{array}{c} \hat{\beta} \\ \hat{\lambda} \end{array} \right] = \left[ \begin{array}{cc} 2 X^T X & C^T \\ C & 0 \end{array} \right]^{-1} \left[ \begin{array}{c} 2 X^T y \\ d \end{array} \right] $$

where $X$ is again the design matrix, $y$ the variable to be explained and $C$ is the restriction matrix such that

$$ C \beta = d. $$

$\lambda$ is in this case the lagrange multipliers. My Question is then, what is the "hat"-equivalent matrix for this type of regression? Will the formulation in $(1)$ hold? My quess is no, since you are adding additional information to the regression. For instance, if you added 0-restrictions to some of the columns, you might as well have excluded them from the design matrix, in which case the leverages would change.

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $h_i$, which is the diagonal of the "hat-matrix", $H$.

In the ordinary least squares case (without linear constraints), this matrix is computed as

$$ H = X\left(X^T X\right)^{-1}X^T \quad \quad \quad (1) $$

Where $X$ is the design matrix and $y$ is the variable to be explained. The constrained least-squares regression i'm attempting to do is the following:

$$ \left[ \begin{array}{c} \hat{\beta} \\ \hat{\lambda} \end{array} \right] = \left[ \begin{array}{cc} 2 X^T X & C^T \\ C & 0 \end{array} \right]^{-1} \left[ \begin{array}{c} 2 X^T y \\ d \end{array} \right] $$

where $X$ is again the design matrix, $y$ the variable to be explained and $C$ is the restriction matrix such that

$$ C \beta = d. $$

$\lambda$ is in this case the lagrange multipliers. My Question is then, what is the "hat"-equivalent matrix for this type of regression? Will the formulation in $(1)$ hold? My quess is no, since you are adding additional information to the regression. For instance, if you added 0-restrictions to some of the columns, you might as well have excluded them from the design matrix, in which case the leverages would change.

Source Link
amri
  • 31
  • 1

How to compute the "hat-matrix" of constrained least squares

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $h_i$, which is the diagonal of the "hat-matrix", $H$.

In the ordinary least squares case (without linear constraints), this matrix is computed as

$$ H = ( X^T X)^{-1}X^T y \quad \quad \quad (1) $$

Where $X$ is the design matrix and $y$ is the variable to be explained. The constrained least-squares regression i'm attempting to do is the following:

$$ \left[ \begin{array}{c} \hat{\beta} \\ \hat{\lambda} \end{array} \right] = \left[ \begin{array}{cc} 2 X^T X & C^T \\ C & 0 \end{array} \right]^{-1} \left[ \begin{array}{c} 2 X^T y \\ d \end{array} \right] $$

where $X$ is again the design matrix, $y$ the variable to be explained and $C$ is the restriction matrix such that

$$ C \beta = d. $$

$\lambda$ is in this case the lagrange multipliers. My Question is then, what is the "hat"-equivalent matrix for this type of regression? Will the formulation in $(1)$ hold? My quess is no, since you are adding additional information to the regression. For instance, if you added 0-restrictions to some of the columns, you might as well have excluded them from the design matrix, in which case the leverages would change.