In my experience it seems me that the interpretation about regression, its meaning and its scope, are debatable and great confusion exist about those things. It seems me that confusions are not go away yet. I started to study regression by Econometrics side and I started to be puzzled, some years ago, thinking about regression and its causal role. My perplexity became both, confirmed and go deeper, when I read: Regression and Causation: A Critical Examination of Six Econometrics Textbooks - Chen and Pearl (2013). Great light was done to me by this article: Shmueli - To explain or to predict - Statistical Science 2010, Vol. 25, No. 3, 289–310. In last article the Author use the word explanation as synonym of causation. To now I’m feel myself more confident about the distinction between prediction and causation. I asked and replied in some related post in this site, as example:
Regression and causality in econometrics
However in his article Shmueli spent some word about description. Unfortunately those word are very few and the only clear thing that we can understand from those are: description is different from both causation and prediction; the role of description are much less important than the others two.
My main question is precisely: what description meaning is? What are the main distinction from description and the other two role for regression? Can you give me a simple as possible example where those differences emerge clearly?
Before to reply, to read the follow seem me needed.
I have found in the web a set of recent slide where Shmueli underscore more clearly that the role of description exist and that is different from others two. To Explain To Predict or To Describe? – satellite conference August 15-16 2019 Kuala Lumpur (Malaysia). You can easily download those. These slide speak precisely about the confusion that again exist in both Academia and Industry about the role of regression. These slide are good. However, unfortunately, too few words are spent about description yet.
I understand that description is markedly different from prediction because, first of all, overfitting and related out of sample measures are the core for prediction while become irrelevant for description scope. However it seem me that dangerous conflating area remain between causation and description. Infact, let me note that a couple of slides was insert for underscore the core difference between: prediction-causation and prediction-description but no one for causation-description.
In my mind I justify, but not appreciate much, the existence of description so: given the variables of interest I estimate regressions in order to reveal pure statistical dependencies. This can be a reply for my question above. This my related question can help (Joint probability distributions and regression)
However, unfortunately yet, at slide 23 Shmueli says something that contradict my position. Because she say that significance (difference from 0) are relevant for the variable to “remain” in the model. Why? It seems me that no dependencies is a respectable descriptive result. Worse, if there are variables selection issue also for description purpose this selection looking for what? It seems me that in this case conflations, mainly toward causation side, hardly will go away. My mistake?
EDIT: Often the regression and in particular the linear regression, that represent the object on which we are most focused here, are presented without the distinctions above or at least them do not emerge clearly. More precisely several presentations leave the impression that once reached a sort of “best specification” (no bias) the regression is ready for any scope and it represent, simultaneously, the best for each of these. All the above precisely contrast this perspective. Regression can be useful for each of three scope above but it must been builded keeping in mind only one. Said that, the differences above come from the scope but the algebra of linear regression remain the same, then the core problem for one scope can play some less important role in any others. This question is strongly related Omitted variable bias vs. Multicollinearity. Moreover, we have to remember that all scope above are though for some kind of “induction/inference”, more data are always better. Basically we observe one sample and would to say something about population or at least something that are not observed yet.
Just here let me note that “regression for description” seems me a misnomer it seems refer on “descriptive statistics”t; instead it is focused on inference about parameters. “Regression for associations” seems me much better as name.
About causation the endogeneity is the main problem, about prediction it is overfitting. At this stage I do not understand clearly if for description particular issue exist. Shmueli suggest multicollinearity and significance of regressors. About the former I’m only partially convinced (see below), about the last I’m almost convinced that is not. From comments was suggested that overfitting, defined as in it, can be problematic for any scope. I'm not convinced about this point also. My motivations follows. They are related to linear regression. More in general the distributional problems and functional form for regression matters but those problem are out of my scope here. Said that:
why endogeneity is not an issue for description? Is true that both are focused on parameters inference. However in causation the concept of bias is related to causal interpretation of at least one parameter while in description the concept of bias not matters. Parameters are unbiased per se in the sense that, stated the existence of conditional expectation and the invertibility in design matrix, parameters are given (as the covariates) and them genuine associational meaning always remain.
Why overfitting is not an issue for description? In linear context overfitting can appear only if we add too many regressors, but we already said that them are given. If we have many regressors and too few data is possible that $R^2$ is much higher than is asymptotic counterpart (overfitting). Similarly is possible that parameters estimators are very imprecise. However we can do nothing. If regressors of interest are given, then, also the best linear approximation of conditional expectation, by linear regression estimated with OLS, are given.
Why (near) multicollinearity only partially is an issue for description? For the same previous motivations. Strongly linear correlation among some regressors are a bad news for inference that is the scope. However we can do nothing if not to separate the collinear regressors and perform several regressions that not suffer from collinearity. Unfortunately in this case the goal is only partially or, at most, indirectly achieved.
