Ways to stabilize OLS betas [closed]

I am estimating the parameters of a system of OLS equations in Matlab. $y=X\beta+\epsilon \to \hat \beta=(X'X)^{-1}X'y$. My $X$ is a $5\times 5$ matrix and $y$ is a $5\times 1000$ matrix, so $\beta$ is also a $5\times 1000$ matrix. The problem is that my parameters are quite unstable as sudden jumps in parameters can occur and they will revert back to normal for the next regression.

My idea for reducing the instability in the $\beta$ is to set a maximum tolerance for $e=y-X\hat \beta$ and find the most stable $\beta$ parameters within this error.

Would you know of any literatures which talks about this that can provide me with the basics? What are some other methods for reducing the parameter instability?

closed as unclear what you're asking by Sycorax, Silverfish, gung♦, kjetil b halvorsen, Nick CoxAug 12 '15 at 20:29

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

• What exactly do you mean by "jumps in parameters"? What is changing that causes the solution to this system to change? Since this obviously is not a set of regression equations (despite the formal appearance), what "literature" are you referring to? – whuber Aug 12 '15 at 16:19

I think you have everything transposed. $X$ should be $m \times n$ where $m$ is the number of observations and $n$ is the number of parameters you're trying to estimate. Similarly, $y$ should be $m \times k$ where $k$ is the number of sets of observations you have (this is typically 1). As a result, $\beta$ will come out to be $n \times k$. In your case, $X$ should be $1000 \times 5$, $\beta$ should be $5 \times 1$ and $y$ should be $1000 \times 1$.
This is just a shortcut for the QR factorization. The QR factorization avoids having to compute $X^T X$, which adds numerically instability.
• Have you considered bootstrapping your data and using the resulting average of the $\beta$ ? – aginensky Aug 12 '15 at 17:12
• I've never heard of factor mimicking regression, so I can't offer you any insight there. But I can tell you this: If the matrix dimensions are correct as stated in your original post, this is not a least squares problem. Rather, you're solving a fully-determined (assuming X is full-rank) linear system with 1000 different right-hand sides. If the solution, $\beta$, changes dramatically with small changes in $y$, that suggests that $X$ is ill-conditioned. Check out the condition number with cond(X). – Bill Woessner Aug 12 '15 at 17:26