4
$\begingroup$

I have been working on a set of data which contains information on the width, age, weight of statues and relate them to the price (I am not actually working on that, but I cannot disclose the topic of my work).

I came up with the following regression:

$Price = -9 -width + 4 log10(age) + 8 height$

the minimum-maximum interval for each parameter are:

width = [0,1[
age = [200,4000]
height = [0.65,0.89]

My price therefore varies between about 4 and 12.

As the value of the constant is -9, which is of the same order of magnitude as my price range, I am wondering if this regression could be criticized for being to generic, with a lot of the price variation unaccounted for and "hidden" inside the constant.

Am I trying to over interpret my data here? Could my dataset be missing a crucial variable, for example the weight?

Improve this question
$\endgroup$
2
  • $\begingroup$ When you say "ranges for each parameter": is that the confidence interval or the min-max for each variable? $\endgroup$
    – AdamO
    Commented Sep 22, 2017 at 13:12
  • $\begingroup$ The min-max interval for each variable $\endgroup$
    – Sorade
    Commented Sep 22, 2017 at 13:12

3 Answers 3

Reset to default
2
$\begingroup$

The simple answer is no. Take a look at the following example. It shows a simple linear regression results between two variables $X$ and $Y$. The only difference between the black points and regression line, and the red ones, is that in the second case I changed $Y$ to $Y + 5$. In the "black scenario, the parameters are $\beta_0 = 0.1256, \beta_1 = 4.9122$, while in the "red" scenario $\beta_0 = 5.1256, \beta_1 = 4.9122$. Answer yourself: did the amount of unaccounted biases changed by five between the two scenarios? Is the "black" model worse because it has lower intercept?

enter image description here

The answer to both questions is "no". The only thing that has changed between the two models is that the mean of $Y$ has changed by five, what has led to intercept changing by the same value. That is exactly the role of intercept: to "shift" the regression line upwards, or downwards, on the $y$-axis, so to correct for the mean of the dependent variable. In fact, in most cases we do not expect the intercept to be anything close to zero, because this would mean regression line going through the origin what in many cases leads to inferior models as compared to the models that include the intercept (i.e. it is non-zero).

I guess that what you are looking for is rather the $R^2$ statistic, but beware that it can be misleading as describes in Is $R^2$ useful or dangerous?

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you Tim, if I understood your answer correctly you are explaining why it is not okay to remove the intercept. I agree with that, my question was more whether the model could be criticised for being to general. Okay I get very good match with my data (adj. R2 > 0.97) but, still, maybe that the large intercept value, means that I don't have enough information to provide meaningful answers. Maybe the data only covered, very light statues. If somebody else was to use the regression for another dataset of heavy statues they wouldn't get a good R2. $\endgroup$
    – Sorade
    Commented Sep 22, 2017 at 13:24
  • $\begingroup$ @Sorade I said nothing about removing intercept, I linked the other thread for you to learn more about the role of intercept in regression. What I say is that intercept is related to the mean of Y and nothing more, it does not measure "bias" and I gave you example of identical models widh different intercepts that changed just because the mean of Y has changed. $\endgroup$
    – Tim
    Commented Sep 22, 2017 at 13:31
  • $\begingroup$ @Sorade please look at the example once again. The models are identical and give identical predictions (that differ by 5), so how could one of them give less information then the another one? $\endgroup$
    – Tim
    Commented Sep 22, 2017 at 13:33
  • $\begingroup$ Thank you. I still feel like I don't have an answer to my question, but you gave me material to think about. I'll brew on it a bit and see if I can maybe rephrase my question better. Maybe my question should be: " If my model has shown good predictability (say I have a divided my data into a test and train set) does it mean that I have accounted for ALL the important parameters for sure ? " $\endgroup$
    – Sorade
    Commented Sep 22, 2017 at 13:47
  • 1
    $\begingroup$ @Sorade The answer to your question in the comment is also "no". First, in most cases it would be impossible to account for ALL the parameters. Second, it could happen that you have similar accuracy metrics for two different models, with different variables. Third, more variables does not mean that your model is going to be "better" (it can however lead to overfitting). In general, it's complicated. You should probably get some handbook on regression modeling if you find this unclear. $\endgroup$
    – Tim
    Commented Sep 22, 2017 at 13:53
2
$\begingroup$

We rarely report or interpret the intercept in a linear regression model. In your case it is an extrapolation of the data. The intercept would be interpreted as a expected price for a product with width 0, an age of 1, and a height of 0. That is nonsense. A value of -9 is an artifact of the projection.

If you want a more interpretable intercept, center the covariates. Then the intercept is the average price for all covariates taking their average values.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you AdamO, your suggestion is interesting. As I am no statistician would you mind giving an example of how you would proceed ? Also, I understand that the "meaning" of the intercept is useless in terms of units of price. Yet the magnitude of it seems large compared to the value of the other terms. $\endgroup$
    – Sorade
    Commented Sep 22, 2017 at 13:28
  • 1
    $\begingroup$ @Sorade you refer to units, but no the units are not useless. The units are (presumably) dollars or revenue or... For -9 to "seem" off you need to become a statistician and look at plots and summary statistics: residuals vs. fitted values. Basically the only way that number is off is if you ran the model incorrectly, which is a software debugging question. $\endgroup$
    – AdamO
    Commented Sep 22, 2017 at 13:32
  • $\begingroup$ Also, if he centers the dependent variable, there will be no intercept at all! $\endgroup$
    – Deep North
    Commented Sep 22, 2017 at 14:03
0
$\begingroup$

Who is the audience that you expect will criticize the intercept? If it is a "business audience", they are much more likely to be concerned about model accuracy, which you haven't mentioned here. The value of the parameter coefficients - showing the relative impact of each variable on price - is where you should focus your interpretation/discussion.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.