Skip to main content
Cross Validated

Return to Answer

added 888 characters in body
Source Link
IMA
IMA
  • 1.2k
  • 5
  • 7

So now you have a pretty good understanding of the problems with the model. You should now go ahead and look at your residuals. Calculate those and look at the plot. Does it look normal? Does it have a systematic error? How are the residuals and the independent variables correlated?
You want to make sure that there is no relation between an independent variable and a residual (at the same datapoint, that is). So plot the correlation of this!
Next you want to make sure your residuals look normal, ie literally normal (distributed). If not, you are probably missing some influence which may make your estimate problematic.
At this point it might be that you have to give up because you just can not make the model work.
However in general you start to add and remove variables, keeping in mind how they correlate with each other. You look at p-values and R-Squared to find a model which explains the influence best. But keep in mind that the p-values only make sense if your residuals are correctly distributed.

I wrote this as a general understanding of some problems. You will have additional issues because of your small sample size, though these are more about accuracy of your forecast (depends). I didn't explain the math or how to technically do this because honestly, that's gonna take quite a bit of research and more than I can or should write here. But now you know what dangers lie along the way!

So now you have a pretty good understanding of the problems with the model. You should now go ahead and look at your residuals. Calculate those and look at the plot. Does it look normal? Does it have a systematic error? How are the residuals and the independent variables correlated?
You want to make sure that there is no relation between an independent variable and a residual (at the same datapoint, that is). So plot the correlation of this!
Next you want to make sure your residuals look normal, ie literally normal (distributed).

So now you have a pretty good understanding of the problems with the model. You should now go ahead and look at your residuals. Calculate those and look at the plot. Does it look normal? Does it have a systematic error? How are the residuals and the independent variables correlated?
You want to make sure that there is no relation between an independent variable and a residual (at the same datapoint, that is). So plot the correlation of this!
Next you want to make sure your residuals look normal, ie literally normal (distributed). If not, you are probably missing some influence which may make your estimate problematic.
At this point it might be that you have to give up because you just can not make the model work.
However in general you start to add and remove variables, keeping in mind how they correlate with each other. You look at p-values and R-Squared to find a model which explains the influence best. But keep in mind that the p-values only make sense if your residuals are correctly distributed.

I wrote this as a general understanding of some problems. You will have additional issues because of your small sample size, though these are more about accuracy of your forecast (depends). I didn't explain the math or how to technically do this because honestly, that's gonna take quite a bit of research and more than I can or should write here. But now you know what dangers lie along the way!

Source Link
IMA
IMA
  • 1.2k
  • 5
  • 7

It is very hard to determine the relationship between variables without having an underlying, prior assumption on how their relationship works. I am not sure about the problem at hand, but seeing that you assume a linear relationship of data points (be it variables or their products) I take it we are looking at a linear relationship.

Indeed there are several problems which may make your regression and forecast downright invalid. Another issue is your small sample size. The result of this is that you have to be even more careful in selecting the right technique.

First, your regression model must be correct for your regression to be correct. This seems tautological, but it entails that no variable should be missing in the equation, if it also has an impact (more correctly, is correlated) with the other variables.
You can immediately see the issue here: It is almost certain that there are more things to winning a cricket game than just the order of batters. For example if the breakfast of the second batter influences the game outcome AND his standing on the batting list, it technically must be in your equation. Otherwise your estimate of the influence of the batting order on the game outcome will be biased, ie wrong.
On the other hand if you have a variable, like weather (maybe), which does not influence the batting order but does influence the game outcome, then your estimate of the influence of the batting order will still be correct. However your forecast of the game outcome BASED on these estimates will be off and should be considered more of a indication than a concrete fact.

You will probably say, well damn, how we ever gonna get a good model then? Good point, almost never will we be exact in this. However there are ways to mitigate the problem or at least diagnose it. We somewhat get around missing variable bias if the missing part is somehow more or less random. In fact we concede anyway that we can not correctly model reality, but as long as the "wrongness" has certain properties, we are still somewhat "right".

The key is that we add the error term in this, a random factor which is normally assumed to be, well, "normal" distributed and centered around zero, though this is already achieved by including a constant variable.
But if we have such an error term in our model, ie. our understanding of reality, then the residuals, which is the difference between the real data and our estimation, should be only created by these error terms and therefore also appear random.
On the other hand if we have a biased estimate because we are missing variables, we can still be confident about our results if the residuals appear to be random around zero.

So now you have a pretty good understanding of the problems with the model. You should now go ahead and look at your residuals. Calculate those and look at the plot. Does it look normal? Does it have a systematic error? How are the residuals and the independent variables correlated?
You want to make sure that there is no relation between an independent variable and a residual (at the same datapoint, that is). So plot the correlation of this!
Next you want to make sure your residuals look normal, ie literally normal (distributed).