Let me explain. You have a multiple regression using time series data within an econometrics model. And, some of your variables are either dummy variables or interaction variables. So, they take a value of 0 or a continuous value. As an example, let's say one of your variable is the quarterly change in 5 year Treasury rate. And, you disaggregate that variable into two. One would cover the pre-Greenspan era from 1951 to 1986. And, the other one would cover the Greenspan era to the present. The first variable will show the quarterly change in such Treasury rates until mid 1986. Thereafter, it will have a value of 0 or 0%. Someone suggested that instead such a variable in the Greenspan era to the present should not have a zero value, but instead be blank. In essence, creating purposefully missing values. The argument is that this variable does not have a true value of 0% in the specified time frame. Instead, it is truly absent. And, this could affect the magnitude of its regression coefficient or impact on Y. I question this rational on two counts. First, I am not sure that any software could calculate such a regression with variables with different numbers of observations. And, I think there is a very specific reason for that as outlined in my second argument. Second, multiple regressions are resolved through matrix algebra including the use of covariance matrices and the inverting and transposing of such matrices. And, those calculations are impossible with missing values on some of the variables. In my mind, I believe I have already answered this question. Can you tell me if this answer is correct. Or am I missing something.
-
1$\begingroup$ Why wouldn't you want to have a single quarterly change in 5 year Treasury Rate column and then a second indicator variable with 0 = pre-Greenspan and 1 = Greenspan? $\endgroup$– WayneCommented Sep 14, 2015 at 17:24
-
$\begingroup$ Your structure creates a Greenspan intercept adjustment dummy variable. There is nothing wrong in doing so. But, it is not the topic of this issue. $\endgroup$– SympaCommented Sep 14, 2015 at 19:05
-
1$\begingroup$ I believe you can also use an interaction to address a slope. Or use a hierarchical model. Both of these suggestions are also not what you've asked for, but it's my feeling that you've spelled out the problems with either specifying 0 or NA, and they seem hard. The NA solution seems equivalent to just running two unpooled regressions (pre- and post-Greenspan). So I'm trying to question whether there isn't a way to structure your problem so you don't have two variables that correspond to the same data but in two different eras, which is the root of your dilemma. $\endgroup$– WayneCommented Sep 14, 2015 at 19:31
-
$\begingroup$ You can run an a model with an interaction variable. The Greenspan intercept adjustment dummy is not statistically significant. So, you can keep the slope-interaction variable 5 year Treasury times Greenspan intercept adjustment dummy (the latter not being used in the regression). And, you get the exact same result than with the variables as structured in example. The meaning of the coefficients are different. But, estimation is identical. $\endgroup$– SympaCommented Sep 14, 2015 at 20:18
-
$\begingroup$ You could set variables to 0 instead of NA, but only if that equates to 0 in your regression formula. For example: X=0 is fine in Y ~ X + sqrt(X), but X=0 is wrong in Y ~ X + log(X). But even the "fine" case is ugly. Don't do it this way! $\endgroup$– CreosoteCommented Sep 14, 2015 at 20:19
2 Answers
It's not that matrix computations are especially finicky: the plain old arithmetical operations they concisely represent are defined on numbers. You can't carry out a regression when an observation has a missing value on a predictor; you need to either substitute a number, or omit the predictor, or omit the observation.
It may assuage someone's doubts to work out the consequences of your coding scheme. For your predictor $x$, let
$$ x_1 =\left\{\begin{array}{ll} x & \mathrm{pre-Greenspan}\\ 0 & \mathrm{post-Greenspan}\\ \end{array}\right.$$
$$ x_2 =\left\{\begin{array}{ll} 0 & \mathrm{pre-Greenspan}\\ x & \mathrm{post-Greenspan}\\ \end{array}\right.$$
The expected value of the response $Y$ is given by the model $$ \operatorname{E}Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots $$ where the $\beta$s are the coefficients you'll estimate. Now pre-Greenspan $$\operatorname{E}Y = \beta_0 + \beta_1 x + \ldots$$ while post-Greenspan $$ \operatorname{E}Y = \beta_0 + \beta_2 x + \ldots $$ —the intercepts are equal & the slopes of $\operatorname{E}Y$ vs $x$ differ in the two time periods, which is, I gather, what you wanted.
As you say, your method is equivalent to @Wayne's method without the indicator variable appearing as an isolated term, but only multiplied by $x$. Note the constraint that when $x$ is zero, $\operatorname{E} Y$ is the same in both time periods (given equal values of other predictors)—most people would want this violation of the marginality principle to have theoretical support as well as not being discrepant with the data.
Why not just calculate a regression for the data pre-Greenspan (using the data 1951-1986) and calculate another regression using the data post-Greenspan (using the data 1986 onwards)? If you have some assumptions on e.g. the intercept, you can still include that. Setting values to zero will naturally distort your regression.
And it is perfectly possible to calculate statistics and perform fitting with missing/incomplete data - just not with the usual matrix formalism (and as said, filling the missing values just to use the "inappropriate" tools is problematic) - in the case of unbalanced observations or missing data the estimators are then calculated as sums (of which matrix multiplication of course is a special case).
-
$\begingroup$ Splitting the regression into two is not interesting to the question at hand. Meanwhile, your second paragraph contradicts the answer of Scorchi. Given that I specified in the question that I was working with a multiple regression framework, I trust Scorchi's answer is correct. And, yours is probably not wrong but not focused on the relevant methodology. $\endgroup$– SympaCommented Sep 22, 2015 at 4:26
-
$\begingroup$ Let me specify: Correct, you cannot do it in a matrix framework. However, as multiple regressions do not always use matrices, this means that you can do multiple regressions in the presence of missing values - you just have to use other tools than matrices to arrive at the result. $\endgroup$ Commented Sep 23, 2015 at 8:51
-
$\begingroup$ Brkn Kybrd can you clarify how you would do that. Using OLS, maximum likelihood, etc..., I am not familiar with a closed form algorithm that could solve for multiple regression coefficients with missing values. $\endgroup$– SympaCommented Sep 23, 2015 at 17:46