0
$\begingroup$
gender = sample(10:100, 10000, replace = TRUE)

desks = sample(0:1, 10000, replace = TRUE)

trees = sample(0:1, 10000, replace = TRUE)

leaves = sample(0:1, 10000, replace = TRUE)

people = sample(0:1, 10000, replace = TRUE)

rebel = c(rep(0, 9999), 1)


df = data.frame(cbind(gender, desks, trees, leaves, people, rebel))

lm = lm(gender ~ ., data = df)

summary(lm)

In this example, we know that rebel has a bunch of 0s and only one 1. If I create a linear model and the p-value of rebel is 0.05, is it wrong to include that variable or to say that the variable's effect is statistically significant?

Should I be removing all columns that only have one 1?

Wouldn't it be misleading if I had a bunch of dummy variables that had a bunch of 0s and they come up as significant on the linear model?

How can we tell if a variable has a 'small sample size' (a bunch of 0s) just by the linear regression summary?

$\endgroup$
2
  • 1
    $\begingroup$ This is the extremal problem of biased data. Going beyond this to having zero samples is really extrapolation. There is a subset of classification called zero shot learning. I would personally look at the intersection of zero shot learning and biased sample fitting. It is a hard hard problem. Personally I would step back and I would start digging into the business driving the question and get a sense of what the answer means so that I can maximize meaningfulness as best as can be done. $\endgroup$ Aug 2 at 16:58
  • 2
    $\begingroup$ It would be interesting to see any version of these data in which that p-value were as low as 0.05. I doubt it's possible. $\endgroup$
    – whuber
    Aug 2 at 17:01

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.