0
$\begingroup$ 
gender = sample(10:100, 10000, replace = TRUE)

desks = sample(0:1, 10000, replace = TRUE)

trees = sample(0:1, 10000, replace = TRUE)

leaves = sample(0:1, 10000, replace = TRUE)

people = sample(0:1, 10000, replace = TRUE)

rebel = c(rep(0, 9999), 1)


df = data.frame(cbind(gender, desks, trees, leaves, people, rebel))

lm = lm(gender ~ ., data = df)

summary(lm)

In this example, we know that rebel has a bunch of 0s and only one 1. If I create a linear model and the p-value of rebel is 0.05, is it wrong to include that variable or to say that the variable's effect is statistically significant?

Should I be removing all columns that only have one 1?

Wouldn't it be misleading if I had a bunch of dummy variables that had a bunch of 0s and they come up as significant on the linear model?

How can we tell if a variable has a 'small sample size' (a bunch of 0s) just by the linear regression summary?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This is the extremal problem of biased data. Going beyond this to having zero samples is really extrapolation. There is a subset of classification called zero shot learning. I would personally look at the intersection of zero shot learning and biased sample fitting. It is a hard hard problem. Personally I would step back and I would start digging into the business driving the question and get a sense of what the answer means so that I can maximize meaningfulness as best as can be done. $\endgroup$
    – EngrStudent
    Aug 2 at 16:58
  • 2
    $\begingroup$ It would be interesting to see any version of these data in which that p-value were as low as 0.05. I doubt it's possible. $\endgroup$
    – whuber ♦
    Aug 2 at 17:01

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.