How to interpret the results of a t test?

Question

I'm currently in an undergraduate statistics course and we are investigating a dataset with R. The dataset is traffic data for roads in the state of Kentucky. We formulated hypothesis about the mean AADT (annual average daily traffic) of the roads in the dataset for 2008 and 2014. I hypothesized that the mean AADT was lower in 2014 than it was in 2008 (as opposed to the null hypothesis that the difference of the means = 0) We then were supposed to perform a two-sample t-test and form a conclusion on the hypotheses using the p-value in context.

I called the following R function to generate the t-test (combined is the name of the dataset):

t.test(AADT ~ year, alt = "less" , data = combined)

And got the results:

## 
##  Welch Two Sample t-test
## 
## data:  AADT by year
## t = 0.20726, df = 198.87, p-value = 0.582
## alternative hypothesis: true difference in means is less than 0
## 95 percent confidence interval:
##      -Inf 4874.681
## sample estimates:
## mean in group 2008 mean in group 2014 
##           10226.00            9682.76

So given the confidence interval and the high p-value, I am inferring that there is weak evidence for rejecting the null hypothesis that the mean AADT of the two years are the same. However, the last line of the output confuses me. How can we not reject the null hypothesis if the data set shows that the mean in 2014 is in fact less than the mean in 2008?

My theory is that this dataset represents traffic calculated from a sample of the roads (since the entire span of each road cannot be observed all the time), and so although we observed a lower AADT in 2014, because this is not the actual AADT, the evidence can still remain weak that the AADT is, in reality, lower than it was in 2009. Do I understand this correctly?

Also, here is a sample of the dataset:

     staID   Route Milepoint District AADT year
1    85787   KY 70     2.900        3 1427 2008
2    85787   KY 70     2.900        3 1193 2014
3 1.90E+24 KY 1120     0.300        6 8084 2008
4 1.90E+24 KY 1120     0.300        6 7985 2014
5    72001  US 641     0.600        1 2610 2008
6    72001  US 641     0.681        1 3006 2014

Answer 1

You must distinguish between descriptive and inferential statistics. Your means are different and this is a descriptive infornation.

Imagine that we somehow would know that in reality bot year had exactly the same traffic. In empirical science this is usually not the case because you can't measure the whole population and because of that you draw a random sample. Even if the true means are the same if you draw a random sample you most probably won't get the exact population mean and, hence, end up with two different means. Descriptive statistics tell you what parameter you observed but inferential statistics tells you how likely it is that the data is a random sample of the null hypothesis (given null is true) and thus the observed differences where just due to sampling error.

In your case this means that you have some descriptive difference between both means but this could have happend just by chance. Or to put it differently: The p value indicates the probability that the difference of the two means (or more extreme mean differences) are observed given that actually the null hypothesis is true and in your case this probability p is 58.2%, which is quite high, hence, the difference between the observed means isn't big enough to discard the null hypothesis that the means are actually equal (in fact, it's not only the difference that might be not big enough becauae it's about the ratio between the mean difference and the standard error, hence increasing the difference or decreasing the standard error can give a significant result).

Anyway, are you sure that you were told to use a two sample t test? Because since you measure the same thing twice (the traffic in Kentucky) I would suggest using a dependent t-test. You maybe want to clarify that.

EDIT

@"For the record, the two-sample t-test was chosen because it appears that the samples from the two years have a different number of entries gives Error (...). It would seem that not all roads were included in both year's samples":

By using the t test as you describe violates the assumption of the t test that the samples are independent. Thus I would suggest using a dependent t test. And the fact that the years don't have same length because you don't have measurements of every road each year simply means that your data has missings, hence, it is not a problem that the two year variables are not of the same length because you can insert NA every time you haven't a value for one road (or impute missings). This gives you two variables of same length. If there are not too many missings using the dependent t test can increase power and maybe the observed difference will be significant.

Answer 2

Your conclusion seems correct to me. It is the difference between unobservable population means being equal which we cannot test for and the differences in sample means being equal.

If the sample means are not equal, that can be either because the population means are not equal, or due to sampling error.

The p-value of 0.582 tells you that 58.2% of the times you would get a difference in sample means as large as you observed or larger, even if the population means are the same. Whether you can reject the null hypothesis depends on the significance level you chose. At $\alpha = 0.05$, you are looking for p-values less than 5%.