2
$\begingroup$

My question is a bit naive. I'm trying to get the exact & clear meaning of the phrase "predictor variables are fixed and not random in linear regression".

According to my understanding, fixed means that we pre-set values of predictors in an experiment and then observe y for those values; whereas random means that while conducting an experiment we first measure the predictor variables and then measure the y (we can do this at different time points to get different Xs and Ys or a dataset).

However even with this understanding of mine (which I am not sure is right), I am not able to understand clearly why do we require the predictive variables to be fixed and not random. What would go wrong if we observe or measure the Xs instead of keeping them pre-set?

TLDR: What is the meaning of fixed X and also the need to keep them fixed?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ The following may provide an answer to your question: stats.stackexchange.com/questions/239185/… $\endgroup$
    – Marjolein Fokkema
    Mar 15, 2022 at 14:55
  • $\begingroup$ Thanks @MarjoleinFokkema! Now it looks like being 'fixed' is related to the errors in measuring X, which would be 0 if X is fixed. (To complement, I also searched through books & found that if the error in measuring X is small as compared to the range of values it lies in, it could be considered as fixed) I'm still a little confused on why exactly is that required - From my understanding that is so to be able to model the relationship between Y & X more accurately; is that right? $\endgroup$
    – SRISHTI GUREJA
    Mar 16, 2022 at 14:44
  • 1
    $\begingroup$ In OLS, we assume residuals and X are orthogonal (unrelated); errors are only related to response Y. For example, errors-in-variables models do not make this assumption (see e.g. wavemetrics.com/products/igorpro/dataanalysis/curvefitting/…). Indeed, I'd expect if error in measuring X is small (and random), the estimated association between X and Y will not be influenced much. $\endgroup$
    – Marjolein Fokkema
    Mar 17, 2022 at 16:38

1 Answer 1

Reset to default
2
$\begingroup$

This terminology refers to whether we want some coefficient in a regression model to be similar but vary across some categories. E.g. if you want to model the relationship between outside temperature and how many minutes flights will be delayed by, you can do a number of things:

  1. assume there is a single slope for all flights $i=1,\ldots,I$, so something like $\text{delay}_i = \text{intercept} + \text{slope} * \text{temperature}_i + \epsilon_i$ (we would call that a fixed effect),
  2. assume that this relationship is different and unrelated for every airport $j=1,\ldots,J$, i.e. $\text{delay}_i = \text{intercept} + \sum_{j=1}^J 1\{\text{airport}_i=j\} *\text{slope}_j * \text{temperature}_i + \epsilon_i$ (we would say that we have separate fixed effects for every airport, or
  3. assume that this relationship is different across airports, but varies across airport according to some distribution (e.g. a normal distribution) so that $\text{delay}_i = \text{intercept} + \sum_{j=1}^J 1\{\text{airport}_i=j\} *\text{slope}_j * \text{temperature}_i + \epsilon_i$ with $\text{slope}_j \sim N(0, \tau^2)$ (we would say that we have a random airport effect on the slope term). Next, you could of course also have a random airport effect on the intercept (or a fixed airport main effect), too (after all some airport just always have longer delays).

Option 3 has the effect that we borrow information from other airports so that airports with few participants have their slope estimates pulled towards the (weighted) average of the slopes for all the airports.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ OP asked why the covariates are fixed, not the coefficients. A design matrix for random effects is still fixed. $\endgroup$
    – Daeyoung
    Mar 15, 2022 at 15:11
  • $\begingroup$ I had suspected that there was a confusion on that terminology, but you may be right that I did not answer the actual question. $\endgroup$
    – Björn
    Mar 15, 2022 at 15:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.