Regression variable has no meaning for one category

For a (binary) logistic regression, I have two IV's in my model. The first IV has three categories (one person, two persons, three or more persons). The second variable is binary (communication exists vs. not exists) For the first category, the second IV has no meaning, but for the second and third category is does.

Question is, how can I include all cases in one regression? The DV is always the same.

This will happen naturally, with no intervention on your part.

Consider, for instance, dummy coding. This system uses vectors of zeros and ones to indicate the categorical variables in a way that allows straightforward interpretation of the coefficients. A variable with $k$ categories is represented by $k-1$ terms (along with an "intercept"). A standard vector notation to describe this uses vector notation.

• The "base" contribution to the response is the intercept $\beta_0$. The corresponding vector is $(1,0,\ldots,0)$ with $k$ components.

• The contribution of the second category relative to the first is $\beta_1$, whence the contribution of the second category is $\beta_0 + \beta_1$. The corresponding vector is $(1,1,0,\ldots,0)$.

• $\cdots$

• The contribution of category $k$ relative to the first is $\beta_{k-1}$, whence the contribution of category $k$ is $\beta_0 + \beta_{k-1}$. The corresponding vector is $(1,0,\ldots,0,1)$.

Thus, each vector has an initial $1$ (for the intercept). The vectors for all categories but the base have a single additional $1$. Each observation, as given by its vector $\mathbf{x}$, contributes

$$\mathbf{x} \cdot (\beta_0, \beta_1, \ldots, \beta_{k-1})$$

to the response. These dot products give the values $\beta_0, \beta_0+\beta_1, \ldots, \beta_0 + \beta_{k-1}$ mentioned in the bulleted list above.

The same system is used when more than one categorical variable is included among the regressors, but they all share the same intercept. In other words, the "base" case is the one where all categorical variables have their base values.

The principal advantage of this coding system--besides being automatic in just about any statistical computing platform--is that the coefficients have simple natural interpretations. To evaluate whether the existence of communication is significant, for instance, you would examine the coefficient associated with $x_2$ ($\beta_3$ in this example) and test whether it differs significantly from zero. This test is usually automatically conducted by software and shown in its summary output.

The question provides a good example. The following table (automatically created by R) shows all six possible combinations of a three-category regressor $x_1$, with values "1", "2", and "3+", and a two-category regressor $x_2$ with values "No" and "Yes".

   x1  x2 Intercept x1=2  x1=3+  x2=Yes  Coefficient
1   No         1    0     0        0  b0
2   No         1    1     0        0  b0 + b1
3+  No         1    0     1        0  b0 + b2
1  Yes         1    0     0        1  b0      + b3 -- there won't be any rows like this
2  Yes         1    1     0        1  b0 + b1 + b3
3+ Yes         1    0     1        1  b0 + b2 + b3


The left two columns show the combined values of $x_1$ and $x_2$. The next remaining four columns correspond to (a) an intercept common to both variables, (b) $3-1=2$ components for the effects of $x_1$ relative to the base, and (c) $2-1=1$ components for the effects of $x_2$ relative to the base (that is, the difference between having communications and not). We may call their coefficients $\beta_0, \beta_1, \beta_2, \beta_3$, in order from left to right. The dot product, showing the contribution of each row to the response, is summarized in the rightmost column (in which b0 stands for $\beta_0$, etc).

When certain combinations are not possible, such as x1=1 and x2=Yes (represented in the fourth row), they simply will not appear in the dataset. Because of this, some might argue that the interpretation of $\beta_3$ should change subtly. Whereas before it would have been understood as the difference between communications and no communications, now it is understood as that difference for the cases where communications make sense.

Here is an example of software output (for a logistic regression) using this coding:

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.65625    0.07841   8.369 3.09e-14 ***
x1.2        -0.33594    0.10373  -3.238  0.00147 **
x1.3+       -0.50781    0.10373  -4.895 2.43e-06 ***
x2Yes        0.04687    0.07841   0.598  0.55085


The four lines correspond to the four similarly-labeled columns in the table. In this case, the software has performed a t-test for x2Yes, which is $\beta_3$, and obtained a p-value of $0.55085$. This would not be considered significant by anyone. The conclusion would be that although there is some evidence that communications increases the chance of a response (as evidenced by the positive estimate $\hat\beta_3 = 0.04687$), it is not significant in this dataset.

• Thanks for this elaboration. Your example in R makes sense. At the moment I use SPSS for the analyses. Would that also be possible? – Bas Oct 9 '15 at 19:11
• SPSS would likely take care of all this automatically, probably in exactly the same manner R does. – whuber Oct 9 '15 at 20:23

If I understand this, what you are saying is that when there is only one person, the question of whether communication exists is meaningless.

You can solve this issue by making the two IVs into one:

1. One person
2. Two people, no communication
3. Two people, communication
4. Three+ people, communication
5. Three+ people, no communication
• Yes exactly. So when I follow your example, it would mean either go for 1 IV with 5 possible values, or for 5 dummy binary variables? – Bas Oct 9 '15 at 13:28
• Although this coding is always possible for any combination of categorical variables, it complicates the process of performing the tests that would be natural in such a situation. – whuber Oct 9 '15 at 14:02
• @whuber you mean 5 dummies complicates? or combining two IV's in general? – Bas Oct 9 '15 at 14:29
• What @whuber is eluding to is that you no longer have a single $\hat{\beta}$ for the communication versus no communication effect or the 3+ people versus 1 person effect. Hypothesis testing for either of these main effects involves joint testing of 2 $\hat{\beta}$s. – BChan Oct 9 '15 at 14:35
• Could one not nest communication in people Y ~ Comms %in% People? I know (think!) it's effectively the same but it might allow more "natural" expression and interpretation. – user20637 Oct 9 '15 at 14:49