Inconsistent P value of one categorical variable in a linear regression and in subgroup analysis Here is a short description of my problem:
The population is randomly assigned to five different treatment groups (placebo, 1g, 2g, 3g or 4g of a drug). We measured subject's body weight at baseline and at 30 days after treatment. I want to see which dose group is effective in weight lose.
I used the following model in R
lm.Model <- lm(weight_D30 ~ weight_baseline + treatment, data = weight_loss)
The "treatment" variable is factor using placebo as the reference.
From this model I can get P values for all treatment arms (with placebo as the reference), for example, the P value for 1g group is 0.04.
However, when I subgroup the population, picking out only placebo and 1g treatment, then put the data into the same model as showed above, the P value is 0.06.
I am wondering if the two different P values (0.04 vs 0.06) imply different hypothesis? What does these P values from the first approach and the second approach really mean?
Thanks very much!
 A: A p-value of 0.04 is not that much different from a p-value 0f 0.06.
But in any event, they are testing the same hypothesis; they just use different data and assumptions. The regression model assumes constant variance across all groups, and uses the pooled variance in the standard error of the difference.  If the constant variance assumption is correct, then the full regression approach is more powerful, as suggested by your smaller p-value for that approach.
On the other hand, if the variances are smaller in the other groups, then the full regression approach can give you standard errors that are too small for the given comparison.  In this case the subset approach is better. because it only assumes constant variance across the two groups.
If you suspect nonconstant variance, you can always incorporate it in your regression model. Then either approach is valid, although the results may still differ slightly.
A: Just to add to the nice explanation by @BigBendRegion of the likely cause, here is a little simulation in R that demonstrates this issue.
We simulate a balanced dataset consisting of two variables x and y where x has 3 levels, A, B and C, with 15 observations per group, and with standard deviations for the residual error of 0.5, 2, and 4 respectively.
library(tidyverse)
set.seed(1)
ea <- 15
dt <- data.frame(x = rep(c("A", "B", "C"), each = ea))
X <- model.matrix(~ x, dt)

betas <- c(10, 1, 1)

dt$y <- X %*% betas + c(rnorm(ea, 0, 0.5), rnorm(ea, 0, 2), rnorm(ea, 0, 4))


Now we fit a linear model to the whoe dataset, and then another after removing the B group:
lm(y ~ x, dt) %>% summary()
dt %>% filter(x != "B") %>% lm(y ~ x, . ) %>% summary()

The estimates for the C group from each model are:
xC            1.3107     0.7148   1.834   0.0738 .    
xC            1.3107     0.7557   1.734   0.0938 .  

The p-value for the model on the reduced dataset is larger becuase the standard error is computed on a larger pooled variance
