0
$\begingroup$

I have some data on a drug where doctors are trying to find out based on collected data how many patients get heart disease because of a drug.

The output is the following:

95%CI 1.72(1.28-2.30) for the patients are taking this drug

95%CI 1.30(1.21-1.40) for the patients which are not taking the drug

Therefore we conclude that there is a 42% absolute increase for getting the heart disease compared to the no users. I think I’m right here?

Back to the CI-95%,, does this mean that if the study was repeated with another sample and the parameters being the same (age, sex, other diseases, other medications taken) that 95% out of the sample (let’s say a sample of 100 people which would then be 95 patients) would show the same amount of people getting the heart disease? Is this what the CI is all about? Because if we take a random sample but not with the same parameters values (age, sex, other diseases, other medications) we wouldn't get even close to the 95% we would have a 20% match or so..

In essence would the other sample yield the result of 95% of the patients being in the 1.28-2.30 interval and the other 5 % being outside of the interval?

Improve this question
$\endgroup$
5
  • $\begingroup$ en.wikipedia.org/wiki/… $\endgroup$
    – Bernhard
    Feb 20, 2017 at 14:59
  • $\begingroup$ @Bernhard "A 95% confidence interval does not mean that 95% of the sample data lie within the interval." Is is then more accurate to say that the sample data should come towards the 95% set interval? Would that be the correct phrasing? $\endgroup$
    – Eugen Sunic
    Feb 20, 2017 at 15:06
  • $\begingroup$ What exactly are the figures you quote? Are they odds? $\endgroup$
    – mdewey
    Feb 20, 2017 at 17:27
  • $\begingroup$ Let's say you conduct your experiment 100,000 times. Each time, you will calculate a different confidence interval. The true population parameter will be inside approximately 95,000 of the confidence intervals. $\endgroup$
    – Matthew Gunn
    Feb 20, 2017 at 19:52
  • $\begingroup$ see point 3. of stats.stackexchange.com/questions/167972/… $\endgroup$
    – user83346
    Feb 21, 2017 at 9:15

1 Answer 1

Reset to default
3
$\begingroup$

No. That isn't correct. Let us imagine you set a 95% confidence interval as your decision rule. Then if you repeated the experiment 100 times then, in expectation, at least 95 of them would have confidence intervals that contain the true parameter. Your current interval either does or does not contain the parameter. You cannot know if it does or does not.

You do not have confidence in the interval, you have confidence in the procedure that creates the interval. There are an infinite number of potential confidence intervals. Any function that covers the parameter at least the specified number of times is a valid interval. Different intervals have different properties, outside their coverage properties.

The edges of the interval are random because they are from a random sample. You would never expect to see the same interval again, under perfect conditions. In fact, the width of the interval tells you nothing about the quality of the research or the precision of the experiment. For example, if you split a room of 100 undergraduates in half, randomly, and took sample statistics then it would irrational to believe that the one with the narrow standard deviation was the better sample. Both were measured using the same experiment, randomization, which should remove any systematic differences leaving only random differences.

If you use an interval in your inference, the most you can say is that your procedure, presuming all other forms of validity have been met, will work in at least 19 of 20 experiments assuming, of course, your number of repetitions is very, very large.

You want to avoid the mental trap of saying that 95% of your data are in that range. That isn't true. You also want to avoid saying that there is a 95% chance the parameter is in the interval. That is not true either. The former error is a variant on the range. The second is an idea not measured here called the credible interval. It's a very different animal.

I don't know how to answer your question on the 42% because you do not describe what you are measuring or its scale.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Not to be nitpicking, but "Then if you repeated the experiment 100 times then at least 95 of them would have confidence intervals that contain the true parameter" is not right, it is on 95% in lots and lots of repeats. No guarantee that it is 95% (or even higher) for a finite number. $\endgroup$
    – Björn
    Feb 21, 2017 at 6:06
  • $\begingroup$ I agree, more like 100,000 repeats. I will edit and rephrase. $\endgroup$
    – Dave Harris
    Feb 21, 2017 at 7:07
  • 1
    $\begingroup$ @Björn that is one of the funny things about confidence intervals, I have yet to see any experiment repeated an infinite number of times, yet we behave as this is a normal idea. $\endgroup$
    – Dave Harris
    Feb 21, 2017 at 7:10
  • $\begingroup$ I'm surprised seeing so many people confused about the CI... $\endgroup$
    – Eugen Sunic
    Feb 21, 2017 at 10:32
  • 1
    $\begingroup$ @eugen_sunic you shouldn't be. It isn't a model of the world even, it is a model of how to model, and it, generally, only works at the limit; there may be some special finite cases I am not aware of or degenerate cases such as when the interval covers the entire sample space. It is a weird idea in the space of all ideas, it does fall into the weird list. I think even Fisher would have put it in the weird list as he was an early critic. $\endgroup$
    – Dave Harris
    Feb 22, 2017 at 3:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.