Question about t-test interpretation

Question

I tried to compare the samples of the movie ratings that cast award winning actors and those that have not casted awarded actors. Then I grouped the averages per year for both awarded and non awarded movies and calculated the mean per year. Then I applied the t.test() function in R for both samples, here are the results:

It looks like we can almost surely say that this higher mean of the sample of awarded movies is not accidental and that movies with award winning cast score higher. But then, for example, the golden globe sample consists only of 76 rows, wouldn't that make this test unreliable? Also, I see that the confidence interval is negative, which seems a bit weird. Does it mean that most years movies had an average score of 5.3 or lower? Also, would it be possible to run randomly generated sample that consists of 76 numbers from 0 to 10 and replicate it 10,000 times and then say whether the globe's mean is just accidental or reliable?

[UPDATE]

Here I did what one user suggested ( My interpretation of the suggestion) and got different t test results.

And my code:

XX <-  replicate(n = 10000, expr = mean(sample(
  x = imdb_winners$averageRating, size = 30, replace = TRUE)))
YY <- replicate(n = 10000, expr = mean(sample(
  x = imdb_not_winners$averageRating, size = 30, replace = TRUE)))

t.test(XX, YY)

Here the t-value is very large so we cal assume that the awarded actors influence the results. I just want to make sure whether my code looks legit and also what could be the best visualisation for these data that could make it appearant that the data are different. Thanks in advice!

Answer 1

My answer will address your questions more specifically, though I have no objections to the response articulated by @AJKOER. You should also investigate these effects in a regression framework. It is more flexible and you can perform a sub-group analysis by year.

It looks like we can almost surely say that this higher mean of the sample of awarded movies is not accidental and that movies with award winning cast score higher.

Yes. The output suggests your results are not due to chance. There is strong evidence that the population mean difference is different from 0. You do not appear partial to any one side, so you care about a difference in either direction.

But then, for example, the golden globe sample consists only of 76 rows, wouldn't that make this test unreliable?

No. The group sample sizes are sufficient. In general, there is no minimum sample size for a $t$-test to be valid. This post should address most of your concerns related to sample size.

Also, I see that the confidence interval is negative, which seems a bit weird.

Your confusion concerns the ordering of your variables inside the t.test() function. In your example, the average rating of the award-winning movies is larger than the non-award winning movies. Based upon your ordering scheme, your difference in sample means is negative, and your confidence interval is bounded around that estimate. In fact, the difference in means is significantly bounded away from zero—to the left. If you reverse the order, your difference in sample means would be positive, which would consequently shift your interval to the right of zero. To demonstrate this explicitly:

$$ \overbrace{6.301449}^{\text{Non-Award}} - \overbrace{7.422947}^{\text{Award}} = \underbrace{-1.121498}_{[-1.236719\hspace{1mm},\hspace{1mm} -1.006276]} $$

In the foregoing example, the 95% confidence interval is entirely to the left of zero. However, if you reverse the order of your samples inside of the function, then your confidence interval will reflect about zero. Reversing the order results in the following:

$$ \overbrace{7.422947}^{\text{Award}} - \overbrace{6.301449}^{\text{Non-Award}} = \underbrace{1.121498}_{[1.006276\hspace{1mm},\hspace{1mm}1.236719]} $$

Variable ordering determines which side of 0 your interval will lie. Since the interval is bounded away from zero to the left, it is also bounded away from zero to the right. You can tell the same story with either approach. You can test this below using a basic simulation in R. See the code at the bottom of this answer.

Does it mean that most years movies had an average score of 5.3 or lower?

Not quite. This is not a test of "most year's movies" in your sample. Rather, you are estimating a difference in two means (i.e., $\bar{X}_{1} - \bar{X}_{2}$). Based upon your output, you can be 95% confident that the true mean difference in movie ratings lies somewhere between -1.236719 and -1.006276 units. I use the term units, but you should tailor it the appropriate rating scheme.

Also, would it be possible to run randomly generated sample that consists of 76 numbers from 0 to 10 and replicate it 10,000 times and then say whether the globe's mean is just accidental or reliable?

R's replicate() function should work just fine. You could do the following:

replicate(10000, t.test(..., ...)),

where each ... is a different sample. This online resource may be helpful!

# Ordering the Sample Means

set.seed(13)
x = rnorm(10)
y = rnorm(10) + 5

t1 <- t.test(x, y)  # x - y
t2 <- t.test(y, x)  # y - x

t1$conf.int  # To the "left" of zero
t2$conf.int  # To the "right" of zero

Answer 2

A different approach, I would reframe your problem into a regression format. In this modeling frame, assign a 0/1 dummy variable to speak as to whether awarded actors have a statistically significant impact on a movie’s rating.

I would take some random sub-averages of the movie rating scores for movies with the same 0/1 dummy variable (relating to quality of actors as having received an award or not). The averaging of the Y data is employed arguably/approximately to introduce normally distributed errors allowing the use of the more powerful t-test and avoiding less powerful non-parametric testing.

I also like the regression format, as it allows one to easily enter possibly other more significant variables, which I believe are more likely generating the rating of movies rather than the nature of the actors. For example, a dummy as to whether or not the movie is very high budget or is actually part of the significant historical money-making sequel. Once you have a dummy for this variable, I believe the population would then reflect more of the market for lower budget movies which have been introduced, online, via Netflix, for example. As the latter has induced some highly awarded creative movies with many unawarded actors, this may be appropriate in assigning the correct rating weight to the unawarded actors not in very large budget or big name sequel movie productions.