This question is primarily for my understanding; say we have a regression equation of the form Y = Xb, where X is a matrix of a few explanatory variables. If you are told that the vector 'b' does not exist, would this equation still have any value? Does it mean we are seeing a non-linear relationship between X and Y?

Thank you!

  • 1
    $\begingroup$ More context, please. You’re always allowed to pick a b vector such that the matrix multiplication works out, so b not existing seems like nonsense to me unless there is additional information. (Such a b chosen at random is unlikely to be useful, sure, but it exists.) $\endgroup$
    – Dave
    Sep 11 at 16:21
  • $\begingroup$ I definitely agree that we need context here (and I think it should have been closed until the question was clarified). However I am confused by the additional comment. If Y and X are given, I am not sure we can always pick a $b$ to make the equation $Y=Xb$ true. How would we do that? Of course if you add in an error term you have lots of d.f. to play with. $\endgroup$
    – Glen_b
    Sep 12 at 0:01

I'm not sure why you would start with the assumption that b doesn't exist.

Regardless of whether or not the relationship between Y and the X variables is linear, it is usually possible to find a value for the vector b (exceptions might be related to multi-collinearity and non-invertibility of X'X).

This is "just" linear algebra: we are finding a set of values such that Y is projected onto the "column space" of X in a way that minimizes the error. A good reference on this is Agresti's Foundations of linear and generalized linear models

So in summary, even if b existed and was calculated, you can't immediately conclude that a linear model is the best option here. On the other hand, if b can't be calculated, you might have to drop variables from X before fitting and trying to assess whether the relationship is linear.

  • $\begingroup$ Thank you for the response! Wouldn't multi-collinearity indicate beta = 1/-1? What would happen if Beta was 0 in this case? $\endgroup$
    – Kiara
    Sep 11 at 16:10
  • $\begingroup$ Hi Kiara, no, I would not expect there to be any reason why multi-collinearity would leat to a beta of 1 or -1. I don't think there is any specific relationship between multi-collinearity and specific values of beta. $\endgroup$
    – Nayef
    Sep 12 at 13:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.