0
$\begingroup$

A little exercise. Let's suppose we are working in company which tell us : our bulbs light have a average duration of 800 hours.

So our null hipotesys is 800 hours.

We make aleatory test of 50 light bulbs and we get:

$$ \bar x = 750 \text{ hours}\\ s = 120\text{ hours}\\ n = 50\\ H_0 = 800\text{ hours}\\ \dfrac{750 - 800}{\frac{120}{\sqrt{50}}} \approx - 2.91 $$

When we looking for in the Z score table we get 2.91 have a probability distribution of 0.0018.

This would be equivalent to 0.18%. So we say we have 0.18% of probability to make type I error, and we say we have very little probability to make a mistake.

Until here all is right. But How to explain this result to low level people ?

QUESTION IS : Is this the same as saying : " We have $99.82\%$ $(100-0.18) $of probability the average of bulbs lights is $750$ hours or less "

Improve this question
$\endgroup$
24
  • $\begingroup$ Use pictures! For example, a number line with the null distribution and empirical distribution plotted together would help your audience. $\endgroup$
    – Galen
    Commented Nov 4, 2021 at 18:54
  • $\begingroup$ "So we say we have 0.18% of probability to make type I error, and we say we have very little probability to make a mistake." This is false. $\endgroup$
    – Dave
    Commented Nov 4, 2021 at 19:05
  • $\begingroup$ @Dave ok. Is false. What is the interpretation ? $\endgroup$
    – NIN
    Commented Nov 4, 2021 at 19:09
  • 1
    $\begingroup$ What's the definition of a p-value? What's the definition of $\alpha?$ // I edited your post to format the math in text instead of a picture. I was tempted to make a mathematical correction, but I think it is best for you to think about the mistake and correct it. Hint: it involves your null hypothesis. Do you see what the mistake is? $\endgroup$
    – Dave
    Commented Nov 4, 2021 at 19:12
  • 1
    $\begingroup$ Then what is the definition of a p-value? // Did you figure out the mistake with your null hypothesis? // I invite you to have the fun of considering how you would respond to a client who says, "You got 750, which is 800-ish. What's the problem?" Even more fun is if you had gotten $\bar x = 799$, particularly if the sample size is gigantic. $\endgroup$
    – Dave
    Commented Nov 4, 2021 at 19:26

1 Answer 1

Reset to default
4
$\begingroup$

The result is hard to explain to non-technical people because it is hard to explain even to statisticians!

The biggest problem is that you are applying the wrong tool incorrectly to the problem that you wish to solve. For a 'hypothesis test' the probability of a false positive is set prior to the analysis as the alpha (or size of the test). It is what Galen is calling a "confidence level". A hypothesis test generates a decision regarding rejection of the null hypothesis but it does not tell you the probability of a light bulb failing early or late, but can be designed to protect you against exceeding the alpha level probability of sending out a batch of bulbs with a mean failure rate lower (or higher) than 800 hours. In other words it can be set up as a useful test for acceptance of the batch of light bulbs.

A hypothesis test does not tell you directly about the probability of a light bulb failure time.

The p-value of 0.0018 that you obtained is the result from a 'significance test', and it says that the observed test results argue fairly strongly against the null hypothesis. It does not tell you the probability of a false positive error because the decision to accept or reject the null hypothesis is not forced by the p-value and so it is dependent on the p-value in combination other considerations. When any decision to reject the null is optional and dependent on factors outside the statistical test procedure the rate of erroneous decisions is not a property of the statistical test. Therefore a significance test does not even have a type I error rate.

Confusion regarding hypothesis test and significance tests is widespread and there are many relevant questions on this site. You can begin here: What is the difference between "testing of hypothesis" and "test of significance"? and here: Interpretation of p-value in hypothesis testing

You might like to report the probability of a light bulb in this batch lasting substantially less (or more) than 800 hours. The tools you want to use for that would not be a hypothesis test or a significance test, but one of several methods that estimate the distribution of light bulb lifetimes. You might like to look at bootstrapping for a very interesting way to do that without having to assume a particular distributional shape.

Improve this answer
$\endgroup$
3
  • $\begingroup$ waoooo, very clear. . .So in my example I just say : there are strong evidence that average of light bulbs is not 800 hours. . . . But I can't say more. $\endgroup$
    – NIN
    Commented Nov 4, 2021 at 20:24
  • $\begingroup$ This would be even harder to explain to non technical people. $\endgroup$
    – NIN
    Commented Nov 4, 2021 at 20:25
  • $\begingroup$ Your data would allow you to say more, but you're right about that test result in isolation. $\endgroup$
    – Michael Lew
    Commented Nov 4, 2021 at 22:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.