5
$\begingroup$

I'm trying to understand Linear Regression. In wikipedia, I was confronted with the following assumption:

Weak exogeneity: This essentially means that the predictor variables x can be treated as fixed values, rather than random variables. This means, for example, that the predictor variables are assumed to be error-free—that is, not contaminated with measurement errors..

My question is regarding the "fixed" characterization of predictors. I don't understand whether X is considered to hold a single value, in the sense represented by the formula $X(Ω) = \{a\}$ where $a$ is a real number, or in the sense that, given a new universe $Ω'$ with a different experiment, $X(Ω')$ holds one value. If the latter was the case, what would the "new" universe be?

I also don't understand why the article claims that "fixed" is the same as being error-free.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

After that wikipedia section it actually further explains:

Although this assumption is not realistic in many settings, dropping it leads to significantly more difficult errors-in-variables models... Because the predictor variables are treated as fixed values (see above), linearity is really only a restriction on the parameters. The predictor variables themselves can be arbitrarily transformed, and in fact multiple copies of the same underlying predictor variable can be added, each one transformed differently. This technique is used, for example, in polynomial regression, which uses linear regression to fit the response variable as an arbitrary polynomial function (up to a given degree) of a predictor variable.

Thus in summary the predictor here is considered fixed against your statistical causal (linear) model. If on the contrary you view the predictor variables as random variables affected by measurement errors which are endogenous variables of your statistical model, then in addition to account for errors in response variables of your linear regression model, you may further need to employ advanced instrumental variables methods or multi-stage methods. Otherwise, simple linear regression would result in an underestimate of the coefficient known as the attenuation bias as summarized in the same reference.

Improve this answer
$\endgroup$
7
  • $\begingroup$ "Thus in summary the predictor here is considered fixed against your statistical causal (linear) model." Does this mean that we consider that each mesure of X is going to render the same result for different individuals, or that each mesure of X for the same individual is going to give the same result (and hence the error-free element)? $\endgroup$
    – Mehdi Charife
    Dec 26, 2022 at 10:37
  • 1
    $\begingroup$ If X is a person's intake of certain pill each day during a month and Y is the same person blood pressure, and your model suppose there's some linear causal relation between them. Fixed against the model (exogeneity) here means we can fully control the value of X without error in an experiment, thus X is not a r.v. in such a model. In observation cases such as X is the years of education and Y is annual income, we assume X is fixed as some parameter in a parameter space. Of course forget about your linear model, each sampled person's years of education is a r.v. which could be modeled further. $\endgroup$
    – mohottnad
    Dec 27, 2022 at 0:14
  • $\begingroup$ Here the regression model is only for the distribution of the r.v. Y given the input X and the model parameters and the randomness is solely from the error term. This is referred to as the discriminative framework. When the inputs are also modelled as coming from a probability distribution, it is referred to as the generative framework. The former is generally more efficient and faster to train, but they can only make predictions for the ranges of training. The latter can make predictions for any range (even not present in the training), but they can be more computationally expensive to train. $\endgroup$
    – mohottnad
    Dec 27, 2022 at 1:43
  • $\begingroup$ "Fixed against the model (exogeneity) here means we can fully control the value of X without error in an experiment" What do you mean by we can fully control the value of X? $\endgroup$
    – Mehdi Charife
    Dec 27, 2022 at 11:22
  • 1
    $\begingroup$ As in my above pill taking experiment, we can control the intake of pills (say 2 pills per day each patient in the experiment group) without any error, thus in such case there's no randomness for the input even viewed outside your hypothesized linear regression model. In your philosophical view/jargon in every possible world the value for either same or different individuals are fixed at 2 (pills/day). This is a perfect case to understand exogeneity of inputs. $\endgroup$
    – mohottnad
    Dec 27, 2022 at 17:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.