# Is it valid to compare p-values from test statistics with different DF?

I need to compare results from data with different residual DF as my $x$ variable has different levels. The following is just an example (in R, for demonstration purpose, but this is not a R question):

# first case
set.seed (123)
data1 <- data.frame (y = rnorm (100, 5, 2),
x = sample (c("A", "B"), 100, replace = T))
anova(lm(y~ x, data = data1))
Analysis of Variance Table

Response: y
Df Sum Sq Mean Sq F value Pr(>F)
x          1   2.07  2.0669  0.6177 0.4338
Residuals 98 327.89  3.3459

# second case:
set.seed (123)
data2 <- data.frame (y = rnorm (100, 5, 2),
x = sample (c("A", "B", "C", "D", "E"), 100, replace = T))
anova(lm(y~ x, data = data2))
Analysis of Variance Table

Response: y
Df Sum Sq Mean Sq F value Pr(>F)
x          4   4.89  1.2224  0.3572 0.8384
Residuals 95 325.07  3.4218


Here I have two different DF for the residuals (95 vs. 96) and $x$ (1 vs. 4): Is it valid to compare p-values as such? I know that the F-test considers $x$ and residual while calculating p-value. Is there any extra-caution needed?

-
The predictors A, B, C, etc. have to mean something. What could you sensibly be comparing? You need to expand you question. What do you mean by compare p-value? Do you mean just make an evaluative judgment on the similarity of findings based on the analysis? –  John Jul 11 '12 at 13:00
yes, comparision is about the judgement –  Ram Sharma Jul 11 '12 at 20:35
And what would you say about the comparison of an analysis of A, B, C, D, E to an analysis of A, B even if you could say something? Let's say the p-value for A, B is lower than all 5 conditions analyzed together. What would that tell you about anything? Are A and B even the same thing in those two cases? –  John Jul 11 '12 at 20:53
A and B can be different, however as anova we are testing null hypothesis that the means of classes are equal ... –  Ram Sharma Jul 11 '12 at 21:05
So, if I do an ANOVA of the influence of artichoke and bean consumption on BMI and another on the presence of aardvarks, bears, coyotes, deer, and elephants on zoo attendance what would it mean if the p-values in ANOVA BMI are lower than ANOVA zoo? –  John Jul 11 '12 at 22:20