0
$\begingroup$

What does the conditional operator (|) do in the linear regression? In the following simulation, I have two sources (s) one of which reports 2 variables (x,z) and the other reports only one variable (x).

library(dplyr)
df1 <- data.frame(x = seq_len(30),  z = c(rep(NA,20), 1:10)) %>%
  mutate(
    s = ifelse(is.na(z), 1, 2),
    y = 2+3*x+5*ifelse(is.na(z), 0, z)
  )

I want the model to read z only when it is available and ignore it otherwise. However when i run lm(y~x+z|s, data = df1) I get only intercept term without the coefficients.

# Call:
# lm(formula = y ~ x + z | s, data = df1)
#
# Coefficients:
#   (Intercept)  x + z | sTRUE  
#         57.67             NA

I also tried lm(y~x+z*s, data = df1) and lm(y~x+z:s, data = df1) but do not get the result I was expecting.

How is this kind data modeled if not with conditional operator or interaction term? Any pointers would be helpful.

$\endgroup$
2
  • $\begingroup$ I assume you know that the "conditional operator" gives the probability distribution for the random variable z when x is a known fixed value. When x is not known the distribution for z could differ from the conditional one. $\endgroup$ Apr 5 '17 at 21:09
  • 1
    $\begingroup$ @MichaelChernick You are correct naturally about standard probability notation, but the question is entirely about R syntax. $\endgroup$
    – Nick Cox
    Apr 5 '17 at 21:37
3
$\begingroup$

| means 'group by' and I'm not sure but I don't think it is used in lm at all. In a mixed effects model, it is used to define the random effect. That is, a different random effect is fitted for each value. e.g.

model <- lme(distance ~ age, Orthodont, random = ~ age | Subject)
ranef(model)

where a different random effect for age is fitted for each Subject.

In the example in your question, you are using it in its other meaning of "OR", so you are including a logical in each observation that evaluates to TRUE if (x+z) or s is greater than 0. This is not what you want (I assume). Your model output is telling you that for all cases where x+z | s evaluates to TRUE has an NA additional intercept.

I think you want the interaction between z and s, which is written z:s. Note the different operator. Note that z*s is shorthand for z + s + z:s which is probably why you were getting answers that you weren't expecting as you are including fixed effects for z, s and the interaction.

$\endgroup$
3
  • $\begingroup$ Strangely x+z:s also gives me the same coefficients as x+z*s. $\endgroup$
    – earthlink
    Apr 6 '17 at 13:10
  • $\begingroup$ Unless you have changed your default na.action option, you are excluding all rows with any NA values (i.e. all of your source 1 in your example). You need to set these to a value (probably zero) to include them. Second, you want s to be a factor df1$s <- factor(df1$s. This will give you an intercept, the coefficient of x the coefficient of z:s1 which is NA given that all z:s1 are zero and the coefficient of z:s2 which is the extra effect of z when source is 2 is in the model. I get 2, 3, NA, 5 for (intercept), x, z:s1, z:s2 in that order (which matches the generator). $\endgroup$
    – Eumenedies
    Apr 6 '17 at 14:51
  • $\begingroup$ Thank you. Coding z to 0 instead of NA gives the expected results. $\endgroup$
    – earthlink
    Apr 6 '17 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.