# When do improper linear models get robustly beautiful?

Questions:

• Are improper linear models used in practice or are they some kind of curiosity described from time to time in scientific journals? If so, in what areas are they used?
• Are there other examples of such models?
• Finally, would standard errors, $p$-values, $R^2$ etc. taken from OLS for such models be correct, or should they be corrected somehow?

Background: Improper linear models are described from time to time in the literature. In general, such models can be described as

$$y = a + b \sum_i w_i x_i + \varepsilon$$

what makes them different from regression is that $w_j$'s are not coefficients estimated in the model, but are weights that are

• equal for each variable $w_i = 1$ (unit-weighted regression),
• based on correlations $w_i = \rho(y, x_i)$ (Dana and Dawes, 2004),
• chosen randomly (Dawes, 1979),
• $-1$ for variables negatively related to $y$, $1$ for variables positively related to $y$ (Wainer, 1976).

Also it is common to use some kind of feature scaling, like converting variables into $Z$-scores. So, this kind of model can be simplified to univariate linear regression

$$y = a + b v + \varepsilon$$

where $v = \sum w_i x$, and can be simply estimated using OLS regression.

References:
Dawes, Robyn M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571-582.

Graefe, A. (2015). Improving forecasts using equally weighted predictors. Journal of Business Research, 68(8), 1792-1799.

Wainer, Howard (1976). Estimating coefficients in linear models: It don't make no nevermind. Psychological Bulletin 83(2), 213.

Dana, J. and Dawes, R.M. (2004). The Superiority of Simple Alternatives to Regression for Social Science Predictions. Journal of Educational and Behavioral Statistics, 29(3), 317-331.

• In what sense would the statistics derived from these models be "incorrect"? – whuber Jul 19 '15 at 13:19
• When the $w_i$s are pre-specified & $b$ estimated, this is just data reduction carried out on the predictors - common enough in various forms (see e.g. the Glasgow Coma Scale & the Charlson Co-morbidity Index) - which won't affect the validity of inference in the usual OLS framework. When $y$ is used to determine the $w_i$s, the standard errors &c. will be out, in the optimistic direction I'd think. – Scortchi - Reinstate Monica Dec 8 '15 at 17:45
• It wasn't an informed comment - the papers are still on my "to read" pile. I just wondered:-"why 'improper'?". It's not unusual for a predictor to be a linear combination of other variables - an average of several measurements, a principal component score, a prediction from another regression, the level from an exponentially smoothed time series, or a calculated value from a well-established or an ad hoc index. Not estimating the weights from the response spares degrees of freedom, helping to avoid over-fitting with smaller sample sizes. – Scortchi - Reinstate Monica Dec 9 '15 at 10:43
• In e.g. Beddhu (2000), "A simple comorbidity scale predicts clinical outcomes and costs in dialysis patients" Am. J. Med., 108, 8 the model equation has the same form as yours where the $x_i$s are defined as the indicator variables for diabetes, lymphoma, & c., & the $w_i$s are pre-specified. I suppose what I'm saying is that the distinction between "improper" & "proper" regression models seems to rest on the notion of a God-given set of $x_i$s, for each of which a "proper" model would estimate a coefficient. – Scortchi - Reinstate Monica Dec 9 '15 at 11:35
• When $w_i = \rho(y, x_i)$, & if $\rho$ were estimated from the same data the model is fit to, that'd be quite a different kettle of fish. – Scortchi - Reinstate Monica Dec 9 '15 at 11:44

This gains in robustness over an ordinary MLR procedure because the number of parameters ($\downarrow$df) is reduced, and introduces inaccuracy because of augmented omitted variable bias, OVB. Because of the OVB, the slope is flattened, $|\hat\beta|<|\beta|$, the coefficient of determination is reduced $\hat{R}^2<R^2$.