1
$\begingroup$

I am interested in using OLS regression to model the relationship between two predictors (X1 and X2) and a response variable (Y). However, for theoretical reasons I know that when X1 = 0, Y must also equal 0. Furthermore, for theoretical reasons I also know Y must be positively related to X1. So I expect a positive correlation between X1 and Y, but I don’t know the value for the slope, or the strength of the relationship. Indeed, what I am most interested in is whether X2 is related to Y, once the variation explained by X1 is taken into account. So I have been considering fixing the intercept to run through the origin. I know from reading other posts that fixing the intercept in this way leads to a number of issues (e.g. high r2 values). However, because I know that the regression line must run through the origin, a model which doesn't do this seems unsatisfactory.

This leads me to my main query – my limited understanding of regression would have me believe that in fixing the intercept at zero, I am also specifying that the intercept for the relationship between Y and X2 is zero (R code: lm(Y ~ 0 + X1 + X2)). Is this right? If so, then then the approach seems flawed, because I don’t have prior knowledge about the relationship between Y and X2. Could people suggest an alternative approach?

Thank you very much in advance for any help, and I hope my question was clear/made sense.

NB The response variable y is bounded between 0 and 1. But it is not the product of trials with discrete outcomes (actually it is the value for an index), so I have been considering using linear regression with angular transformation of the response, rather than a binomial glm.

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ If you really require that $X_1 = 0$ implies $Y = 0$, regardless of the value of $X_2$, then linear regression isn't the tool for you; the only linear equations satisfying that constraint are of the form $Y = \beta_1 X_1$, no $X_2$ term at all. $\endgroup$ – Dougal Jul 24 '15 at 22:05
  • $\begingroup$ There are all kinds of ways to cope with this using linear regression, but the very first thing you should do is include the constant in a multiple regression to see whether it differs significantly from zero in the first place. What results do you get? $\endgroup$ – whuber♦ Jul 24 '15 at 22:19
  • $\begingroup$ Thanks for your reply - yes the intercept is significantly different from zero. $\endgroup$ – Esteban Jul 24 '15 at 22:44
  • $\begingroup$ Are you using linear regression to approximate a deterministic relation (in which case you assert that $Y = f(X_1),\; f(0) = 0$), or the relation is stochastic (in which case you only assert that $E(Y\mid X_1=0) = 0$? $\endgroup$ – Alecos Papadopoulos Jul 25 '15 at 12:37
3
$\begingroup$

As @Dougal was pointing out, by excluding the intercept, you have forced

$\hat Y = 0$ if $X_1 = X_2 = 0$, which is not the same as forcing $\hat Y = 0$ if $X_1 = 0$.

Further more, one should be very, very cautious in deciding to exclude an intercept, even if we are sure that $E[Y|X = 0] = 0$ (let's just consider one covariate for a moment to simplify this explanation). For example, suppose we know that $E[Y|X = 0] = 0$, but the relation between $X$ and $Y$ is not exactly linear. Then if we include an intercept (that may then become statistically significantly different than 0), at least we have a first order approximation of the relation between $X$ and $Y$, centered about the mean of our $X$. We should be wary about making estimates for values of $X$ that are far from the center, but we should already be afraid of that anyways (i.e. extrapolating), and we should get reasonable estimation for values of $X$ relatively close to it's mean. But if we leave out the intercept and have a non-linear relation, the only values of $X$ where we can expect reasonable estimation is close to $X = 0$ (and even then, we don't actually have an approximation of the change in $Y$ given $X$ when $X$ is close to 0 unless most of our data was centered around 0). This is then a waste of effort, because apparently we already knew that $E[Y|X = 0] = 0$ for some reason.

Plus, estimating an intercept only costs you one degree of freedom, so there's not much loss in estimating it. If your relation between $X$ and $Y$ is truly linear, and $E[Y| X = 0] = 0$ is really true, by not estimating the intercept, we basically get one more observation. Not a big deal for reasonably sized datasets (especially when compared to the cost of robustness to assumptions).

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.