0
$\begingroup$

I'm currently in an undergraduate statistics course and we are investigating a dataset with R. The dataset is traffic data for roads in the state of Kentucky. We formulated hypothesis about the mean AADT (annual average daily traffic) of the roads in the dataset for 2008 and 2014. I hypothesized that the mean AADT was lower in 2014 than it was in 2008 (as opposed to the null hypothesis that the difference of the means = 0) We then were supposed to perform a two-sample t-test and form a conclusion on the hypotheses using the p-value in context.

I called the following R function to generate the t-test (combined is the name of the dataset):

t.test(AADT ~ year, alt = "less" , data = combined)

And got the results:

## 
##  Welch Two Sample t-test
## 
## data:  AADT by year
## t = 0.20726, df = 198.87, p-value = 0.582
## alternative hypothesis: true difference in means is less than 0
## 95 percent confidence interval:
##      -Inf 4874.681
## sample estimates:
## mean in group 2008 mean in group 2014 
##           10226.00            9682.76

So given the confidence interval and the high p-value, I am inferring that there is weak evidence for rejecting the null hypothesis that the mean AADT of the two years are the same. However, the last line of the output confuses me. How can we not reject the null hypothesis if the data set shows that the mean in 2014 is in fact less than the mean in 2008?

My theory is that this dataset represents traffic calculated from a sample of the roads (since the entire span of each road cannot be observed all the time), and so although we observed a lower AADT in 2014, because this is not the actual AADT, the evidence can still remain weak that the AADT is, in reality, lower than it was in 2009. Do I understand this correctly?

Also, here is a sample of the dataset:

     staID   Route Milepoint District AADT year
1    85787   KY 70     2.900        3 1427 2008
2    85787   KY 70     2.900        3 1193 2014
3 1.90E+24 KY 1120     0.300        6 8084 2008
4 1.90E+24 KY 1120     0.300        6 7985 2014
5    72001  US 641     0.600        1 2610 2008
6    72001  US 641     0.681        1 3006 2014
|cite|improve this question
$\endgroup$
  • $\begingroup$ I think you should try separate tests, one for each district 1, 3, 6. In district 1 there was increased traffic, decreased in the other two. Putting the three districts together might hide something interesting. // Your table should explain the strange ID for district 6 (or leave out IDs). Is there a way to show variability for each Yr/Dist combination as well as AADT? $\endgroup$ – BruceET Apr 21 '19 at 21:42
  • $\begingroup$ @BruceET I created subsets for each district and repeated the t-tests to try what you suggested and still found weak evidence to reject the null hypothesis (high p-value, zero in CI) in each one. $\endgroup$ – irowe Apr 21 '19 at 22:06
  • $\begingroup$ Well then, 'weak evidence' it must be. $\endgroup$ – BruceET Apr 21 '19 at 22:25
0
$\begingroup$

You must distinguish between descriptive and inferential statistics. Your means are different and this is a descriptive infornation.

Imagine that we somehow would know that in reality bot year had exactly the same traffic. In empirical science this is usually not the case because you can't measure the whole population and because of that you draw a random sample. Even if the true means are the same if you draw a random sample you most probably won't get the exact population mean and, hence, end up with two different means. Descriptive statistics tell you what parameter you observed but inferential statistics tells you how likely it is that the data is a random sample of the null hypothesis (given null is true) and thus the observed differences where just due to sampling error.

In your case this means that you have some descriptive difference between both means but this could have happend just by chance. Or to put it differently: The p value indicates the probability that the difference of the two means (or more extreme mean differences) are observed given that actually the null hypothesis is true and in your case this probability p is 58.2%, which is quite high, hence, the difference between the observed means isn't big enough to discard the null hypothesis that the means are actually equal (in fact, it's not only the difference that might be not big enough becauae it's about the ratio between the mean difference and the standard error, hence increasing the difference or decreasing the standard error can give a significant result).

Anyway, are you sure that you were told to use a two sample t test? Because since you measure the same thing twice (the traffic in Kentucky) I would suggest using a dependent t-test. You maybe want to clarify that.

EDIT

@"For the record, the two-sample t-test was chosen because it appears that the samples from the two years have a different number of entries gives Error (...). It would seem that not all roads were included in both year's samples":

By using the t test as you describe violates the assumption of the t test that the samples are independent. Thus I would suggest using a dependent t test. And the fact that the years don't have same length because you don't have measurements of every road each year simply means that your data has missings, hence, it is not a problem that the two year variables are not of the same length because you can insert NA every time you haven't a value for one road (or impute missings). This gives you two variables of same length. If there are not too many missings using the dependent t test can increase power and maybe the observed difference will be significant.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for the response, this adds a lot of clarity. For the record, the two-sample t-test was chosen because it appears that the samples from the two years have a different number of entries (running t.test(AADT ~ year,paired=TRUE, alt = "less" , data = combined) gives Error in complete.cases(x, y) : not all arguments have the same length). It would seem that not all roads were included in both year's samples. $\endgroup$ – irowe Apr 21 '19 at 21:26
  • $\begingroup$ @stats.and.r, welcome to CV. Unfortunately, this answer is also incorrect. The $p$-value is not a probability of the null-hypothesis being true. In fact, no frequentist measures of evidence for a hypothesis exist. $\endgroup$ – Frans Rodenburg Apr 22 '19 at 3:49
  • $\begingroup$ @Frans Rodenburg: I wasn't saying that. What exact sentence do you interpret that way? I have an understanding of p value as you also can see here where I answered a question about p values stats.stackexchange.com/questions/404119/… $\endgroup$ – user244721 Apr 22 '19 at 4:21
  • $\begingroup$ This sentence: "The probability that the two means have actually a difference of zero is 58.2%" reads as: "The probability that [the null hypothesis is true] is 58.2%. $\endgroup$ – Frans Rodenburg Apr 22 '19 at 4:30
  • $\begingroup$ I tried to clarify that sentence. $\endgroup$ – user244721 Apr 22 '19 at 8:25
1
$\begingroup$

Your conclusion seems correct to me. It is the difference between unobservable population means being equal which we cannot test for and the differences in sample means being equal.

If the sample means are not equal, that can be either because the population means are not equal, or due to sampling error.

The p-value of 0.582 tells you that 58.2% of the times you would get a difference in sample means as large as you observed or larger, even if the population means are the same. Whether you can reject the null hypothesis depends on the significance level you chose. At $\alpha = 0.05$, you are looking for p-values less than 5%.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for the answer, that makes sense. One question: wouldn't a p-value of 0.582 mean that 58.2% (as opposed to 5.82%) of the time I would get the sample difference I got even if the null hypothesis were true? $\endgroup$ – irowe Apr 21 '19 at 21:15
  • $\begingroup$ @irowe, That is correct. I'm assuming it was a typo, so I edited it. This is in fact the only correct answer. $\endgroup$ – Frans Rodenburg Apr 22 '19 at 3:46
  • $\begingroup$ @FransRodenburg Thanks for the corrections. It is indeed 58.2%. $\endgroup$ – norbertk Apr 22 '19 at 10:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.