2
$\begingroup$

I am in the process of working through the Udacity course Intro to Inferential Statistics. In Lesson 9: Hypothesis Testing, the following example is given.

Say that we record survey data from all students of this Udacity course detailing how engaged they are by the lecture materials. We find that the population mean engagement level (on a scale of 1-10) is 7.47 with a standard deviation of 2.41.

The instructor decides to sing a song in one of the lectures to try to spark engagement, and we want to know if the average engagement level changes once the song has been implemented.

We collect a sample size of n=30 and find it to have $\bar{X} = 8.3$. We are asked to perform a two-tailed z-test with $\alpha = 0.05$ to find if the mean is affected by the song.

Additionally, we are given the information that $\mu_{song} = 7.8$, i.e. that the population of all students who listened to the song had an average engagement level of 7.8. We are asked to find if the z-test provides a result that accurately reflects the population, and if not, whether a type I or type II error occurred.

I performed a z-test on the sample and came to a z-score of 1.88, which is less than the critical value of 1.96. Since $\mu_{song} = 7.8$ which is greater than $\mu = 7.47$, I determined that we fail to reject the null hypothesis when it is false (since $\mu_{song} > \mu$) and therefore that type-II error is occurring.

However, this answer was returned as incorrect. Instead, the instructor explained that a z-score needs to be calculated for $\mu_{song}$ as well, equivalent to $\frac{\mu_{song} - \mu}{\frac{\sigma}{\sqrt n}} = \frac{7.8 - 7.47}{\frac{2.41}{\sqrt 30}} = .75$. Since .75 does not fall beyond the critical value of 1.96 at $\alpha=0.05$, $\mu_{song}$ is not significantly different from $\mu$, supporting the null hypothesis. Therefore, we fail to reject the null hypothesis when it is indeed correct.

I don't understand this answer. I would have believed that since $\mu_{song} \neq \mu$, that in reality we should be rejecting the null hypothesis (that they are equal), and that type-II error was occurring. Additionally, we are never given a sample size for the population, so I'm not sure where n=30 came from.

Could someone clarify the correct approach to a problem like this? If you are in a situation where you know the population parameters for the intervened population, does an additional z-score need to be calculated? Or is the fact that the intervention mean is different from the original population mean enough evidence that the null hypothesis ($\mu = \mu_{I}$) is false?

If you are interested in trying out the question yourself for better understanding, you can find it here.

$\endgroup$
2
$\begingroup$

I just watched the first part of that. That ... worries me.

Short answer: Unless more context than I just had drastically alters what they're doing there, then you're right in thinking that they're wrong.

Let me come at this from two different directions:

1) The point of this kind of inference is to make statements about unknown population quantities. If you know the population quantities, then you don't need inference at all. You know what the $\mu$'s are and they're different, end of story.

2) A population quantity (like $\mu$) has standard error 0. It's a known constant.

Consequently, $\mu-\mu_{\text{song}}$ also has standard error 0. If you know both $\mu$'s it's also a known constant.

It makes no sense to attempt to do it, but if they insist, the correct value of $\frac{\mu-\mu_{\text{song}}}{\text{se}(\mu-\mu_{\text{song}})}=\frac{0.33}{0}$. The number of standard deviations those two are apart is infinite.

Why are they computing its standard error as $\sigma/\sqrt{30}$?? I don't know.

Why are they treating $\mu-\mu_{\text{song}}$ as a random variable? Beats me.

Either they're totally wrong or they're trying to do something that's not wrong but talking about it in a completely misleading/wrong way (such as flouting conventions about what 'population' and 'sample' are for example, and how to denote them).

I do hope they come along and explain what they intended, because if it's not as bad as it seems they really should get to explain what was going on there. (I'd be happy to engage in a chat with someone and would be keen to revise this answer if I am shown to have misunderstood anything. Or a clarification in comments would be fine, it doesn't much matter.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.