# Coefficient p-values for linear regression on large dataset

I am running an experiment. There are two features: Gender and Area.

Gender has two levels (Male/Female) and Area has four (North/South/East/West).

The dependent variable is height. My sample size is really big (about a million people).

My goal is to find out whether/how much of the difference in height between men and women is due to the area they live in.

Because the data groups are of different sizes, and I can't guarantee normality of distribution of homogeneity of variance, I didn't think I could use ANOVA. Instead, I was trying to use linear regression with dummy variables and check whether the interaction terms were significant. I got this idea by reading https://en.wikipedia.org/wiki/Dummy_variable_%28statistics%29#Interactions_among_dummy_variables.

Is this the right approach, and if so, how do I find out whether the individual coefficients are significant if I have a really large data set? I have read about some examples involving popular statistics packages that gave a std. error, t-statistic and p value, but given how much data I am using I wasn't sure they would be able to handle it. I know there are faster ways to get (or at least to accurately estimate i.e. stochastic gradient descent) just the coefficients, but I don't know about the standard error.

• ANOVA and linear regression using dummies are identical, with identical assumptions. – Glen_b Jul 4 '13 at 2:58
• The extent to which this could be described as an experiment is an important detail. Even taking gender as given, area is standing proxy for an enormous number of possible influences including climate, nutrition, standard of living, age and race composition. (You don't say which country this is, but the bigger the country, the more the problems. I guess the US.) I think @Russell S. Pierce is hinting in this direction too. – Nick Cox Jul 4 '13 at 14:22

There are other approaches, but without random assignment you are going to have a hard time assigning causality no matter what statistical approach you use. You can find out the degree of relationship between area lived in and height. Using regression (in this context) does not free you from the assumption of normality of residuals. Given your categorical variables ANOVA may be simpler (although regression can perform as an ANOVA). You can always go back and do post-hocs that do not make the homogeneity of variance assumption if you are very worried about it (e.g. Welch's t tests with appropriate p-value corrections for multiple comparisons). With the sample size you are talking about I wouldn't be worried about statistical power (making Type-II errors) just because practically significant effects on height are likely to survive even brutal p-value corrections.

You can use dummy variables and look at the interaction. Each coefficient should have a significance value attached to it which tells you about the comparison between that group and the groups you coded as 0. For example, if Area 1 was coded as 0 and Male was coded as 0 then the coefficient for Gender x Area2 would tell you if the effect of gender in Area 2 differed from the effect of gender in Area 1... and so on. For simplicity to answer all of your potential questions you may need to recast the equation a couple times selecting different dummy codes.

A million cases might seem like a lot, but non-student versions of most mainline statistical packages will be able to handle a million cases without any problem (citations: SPSS, SAS, R). I know that doesn't really address the tail end of your question there, but for now my answer is that the premise is wrong.

You seem to be concentrating on statistical significance. When you have a large data set, even very small differences will be statistically significant. Statistical significance simply means "If, in the population from which this sample was drawn there really was no effect, how likely is it to get a test statistic at least as extreme as the one we got in a sample the size of the one we had?"

You should probably be concentrating instead on the size of the coefficients and whether they are of any practical importance.

Here is how your model may look like:

$\text{Height}=\beta_{0}+ \beta_{m}\text{Male}+\beta_{s}\text{South}+\beta_{e}\text{East}+\beta_{w}\text{West}\\ \quad\quad\quad\quad+\beta_{ms}\text{Male}\times\text{South} + \beta_{me}\text{Male}\times\text{East}+\beta_{mw}\text{Male}\times\text{West}+ \text{error term}$

Interpretation: $\beta_{m}$ gives you the difference of heights between male and female, $\beta_{s}$ gives you the difference of heights between South and North, and $\beta_{ms}$ gives you the difference of heights between male and female for South over North. Similar interpretation holds for other coefficients. Your focus should be on the significance of interaction terms.

Note: Male takes a value of 1 for male and 0 otherwise; South takes a value of 1 for South and 0 otherwise and so on. Female and North act as base here.

I think you won't have the problem computing that in R or Stata.

• This is what I was thinking, but I assumed that a million examples would be too many. Maybe I am just too inexperienced with R/Stata! – Andrew Latham Jul 4 '13 at 4:24
• May be you can get better answer from SO on your second issue – Metrics Jul 4 '13 at 12:23