QR Factorization to Solve Least Squares Without Using an Inverse

Question

I'm playing around with different ways to solve least squares, and am using numpy to derive values for $\beta$ in a regression problem.

I know that if you do a $QR$ factorization of $X$ such that $ X = QR $ where Q is an $m x n$ orthonormal matrix and $R$ is an $n x n$ upper triangular matrix, then you can derive $\beta$ by:

$\beta$ = $R^{-1}Q^{T}y$.

In numpy this looks like this:

beta = np.linalg.inv(R).dot(Q.T.dot(y))

However, my understanding is that, from an optimization standpoint, it's always a bad idea to take the inverse of a matrix.

So, if one wanted to do a QR factorization to derive the correct values of $\beta$, then how would one do this without taking the inverse of $R$?

Answer 1

First, observe that $R \beta = Q^\top y$ involves a triangular matrix $R$, which is easy to solve for $\beta$ without forming an explicit inverse by using forward substitution or backward substitution.

In python, we can solve this using the specialized triangular system solver:

beta = scipy.linalg.solve_triangular(R, Q.T.dot(y))