For a dataset, I have fitted a model. The fitted or predicted values have less variability than the observed values. What does it imply?
$\begingroup$
$\endgroup$
7
-
$\begingroup$ That your model explains a part of the observed variability. $\endgroup$– SergioCommented Sep 12, 2020 at 14:09
-
$\begingroup$ @Sergio Is it bad indication that the fitted values are not as spread as observed values? $\endgroup$– user149054Commented Sep 12, 2020 at 14:34
-
$\begingroup$ In the 1880's, Francis Galton discovered this was a universal phenomenon. It is part of his theory of "regression to the mean." He illustrated it with his celebrated quincunx. $\endgroup$– whuber ♦Commented Sep 12, 2020 at 14:56
-
1$\begingroup$ Variability of observed values: $\sum(y_i-\bar{y})^2$. Variability of fitted values: $\sum(\hat{y}_i-\bar{y})^2$. Variability around your model: $\sum(y_i-\hat{y})^2$. Since $$\sum(y_i-\bar{y})^2=\sum(y_i-\hat{y})^2+\sum(\hat{y}_i-\bar{y})^2$$ the variability of fitted values can never be greater than the variability of observed values, and they can be equal only if you model explains absolutely nothing. $\endgroup$– SergioCommented Sep 12, 2020 at 15:01
-
1$\begingroup$ @InColorado: The concept has a name, and it is "analysis of variance". See Kutner, Nachtsheim, Neter & Li, Applied Linear Statistical Models, §2.7; Weisberg, Appplied Linear Regression, §2.6; Draper & Smith, Applied Regression Analysis, §1.3; Graybill & Iyer, Regression Analysis, §3.8 etc etc $\endgroup$– SergioCommented Feb 5, 2021 at 8:36
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
It is to be expected that the variability in fitted values is less than the variability in observed values. The lower the variability in fitted values, the "better" the model. When you fit a model to data you are trying to "explain" the variability in the data. There will almost always be some random error due to natural variation, or measurement error, for example, so your model can never be expected to explain all the variation. In the comments, sergio has put it very well:
Variability of observed values: $\sum(y_i-\bar{y})^2$. Variability of fitted values: $\sum(\hat{y}_i-\bar{y})^2$. Variability around your model: $\sum(y_i-\hat{y})^2$. Since $$ \sum(y_i-\bar{y})^2=\sum(y_i-\hat{y})^2+\sum(\hat{y}_i-\bar{y})^2$$ the variability of fitted values can never be greater than the variability of observed values, and they can be equal only if you model explains absolutely nothing
answered Sep 13, 2020 at 8:03