I have a data with 20k samples. It includes 6 categorical and 2 continuous variables. When I performed multiple linear regression and ANOVA to test the variables significance, it shows p-values 1e-16 for all variables. I suppose it is because of data size and has to do something with power. However, I am quite sure that 2 varables are not important for the regression at all. Could I use sum of squares as a quatitative measure of significance instead of p-values or something else? I can also reduce the size of data, but how much? What is an optimal size which I should use?
If you do an ANOVA, you can see the sum of squared variance explained by each variable. This will help you judge how much explanatory power is added.
Here's some sample R code:
tmp <- data.frame(a = rnorm(100), b = rnorm(100)) tmp$c = tmp$a + 3*tmp$b + rnorm(100) anova(lm(c ~ a + b, tmp))
Alternatively you can look at the R-squared of regression models with and without the relevant variables, and take the difference. This tells you how much extra explanatory work the variables are doing. I think the two approaches are almost identical.