Skip to main content
Cross Validated

Return to Answer

expanding
Source Link
Pere
Pere
  • 6.6k
  • 1
  • 18
  • 34

Your numbers

To answer to the comment I will try to show that the intersection approach doesn't work, but before going to the general case I'd like to discuss the numbers in the question (readers interested just in the general case may want to skip this section).

From your confidence intervals I assume that you took a first sample of about 600 observations and got a market share of 33% and several months later you took another sample of 600 observations and got a 39% market share. The combined sample (your boosted sample) gives total of 36% market share. I assume that you are using a 90% confidence level, which is unusually low (you are getting narrower but less reliable confidence intervals than usual).

The confidence intervals for your two samples and for the combined sample are:

> prop.test(x=0.33*600,n=600,conf.level=.9)
data:  0.33 * 600 out of 600, null probability 0.5
90 percent confidence interval:
 0.2984402 0.3631262

> prop.test(x=0.39*600,n=600,conf.level=.9)
data:  0.39 * 600 out of 600, null probability 0.5
90 percent confidence interval:
 0.3569938 0.4240188

> prop.test(x=0.36*600*2,n=600*2,conf.level=.9)
data:  0.36 * 600 * 2 out of 600 * 2, null probability 0.5
90 percent confidence interval:
 0.3371367 0.3835047

You can see that the first and third intervals match the intervals in the question - that's how I guessed the data.

The first question should be whether marked share has changed in those months. A not very good sign is that the first and second interval nearly don't overlap, but we should perform a test:

> prop.test(x=c(0.33*600,0.39*600),n=c(600,600),conf.level=.9)

    2-sample test for equality of proportions with continuity correction

data:  c(0.33 * 600, 0.39 * 600) out of c(600, 600)
X-squared = 4.4307, df = 1, p-value = 0.0353
alternative hypothesis: two.sided
90 percent confidence interval:
 -0.10716107 -0.01283893
sample estimates:
prop 1 prop 2 
  0.33   0.39

You can see that the test rejects equality of the two proportions and with the same confidence level you used it says that market share has been increased between 1.28% and 10.71% (if brand X is your client that should be good news).

Overlapping intervals

If market share hadn't changed, you would have two independent samples (before and after) and a combined sample.

The 90% confidence built on each independent sample has a 90% probability of containing the true market share. Since both are independent, the probability of both containing the true market share is 0.9*0.9=0.81=81%. Then, if we take the intersection we just have a 81% confidence interval, which is even lower.

If we take the first sample and the combined sample, the probability of the true market share being in each is 90%, but since they are not independent the probability of it being in the intersection will be somewhere between 81% and 90%.

If the true market share has changed in the months between both samples, the situation is still worse, since the intersection is shrinking just because the two intervals are moving. The intersection is not being narrower because we are getting more precision, it's getting narrower because we are measuring two different market shares.

Your numbers

To answer to the comment I will try to show that the intersection approach doesn't work, but before going to the general case I'd like to discuss the numbers in the question (readers interested just in the general case may want to skip this section).

From your confidence intervals I assume that you took a first sample of about 600 observations and got a market share of 33% and several months later you took another sample of 600 observations and got a 39% market share. The combined sample (your boosted sample) gives total of 36% market share. I assume that you are using a 90% confidence level, which is unusually low (you are getting narrower but less reliable confidence intervals than usual).

The confidence intervals for your two samples and for the combined sample are:

> prop.test(x=0.33*600,n=600,conf.level=.9)
data:  0.33 * 600 out of 600, null probability 0.5
90 percent confidence interval:
 0.2984402 0.3631262

> prop.test(x=0.39*600,n=600,conf.level=.9)
data:  0.39 * 600 out of 600, null probability 0.5
90 percent confidence interval:
 0.3569938 0.4240188

> prop.test(x=0.36*600*2,n=600*2,conf.level=.9)
data:  0.36 * 600 * 2 out of 600 * 2, null probability 0.5
90 percent confidence interval:
 0.3371367 0.3835047

You can see that the first and third intervals match the intervals in the question - that's how I guessed the data.

The first question should be whether marked share has changed in those months. A not very good sign is that the first and second interval nearly don't overlap, but we should perform a test:

> prop.test(x=c(0.33*600,0.39*600),n=c(600,600),conf.level=.9)

    2-sample test for equality of proportions with continuity correction

data:  c(0.33 * 600, 0.39 * 600) out of c(600, 600)
X-squared = 4.4307, df = 1, p-value = 0.0353
alternative hypothesis: two.sided
90 percent confidence interval:
 -0.10716107 -0.01283893
sample estimates:
prop 1 prop 2 
  0.33   0.39

You can see that the test rejects equality of the two proportions and with the same confidence level you used it says that market share has been increased between 1.28% and 10.71% (if brand X is your client that should be good news).

Overlapping intervals

If market share hadn't changed, you would have two independent samples (before and after) and a combined sample.

The 90% confidence built on each independent sample has a 90% probability of containing the true market share. Since both are independent, the probability of both containing the true market share is 0.9*0.9=0.81=81%. Then, if we take the intersection we just have a 81% confidence interval, which is even lower.

If we take the first sample and the combined sample, the probability of the true market share being in each is 90%, but since they are not independent the probability of it being in the intersection will be somewhere between 81% and 90%.

If the true market share has changed in the months between both samples, the situation is still worse, since the intersection is shrinking just because the two intervals are moving. The intersection is not being narrower because we are getting more precision, it's getting narrower because we are measuring two different market shares.

Source Link
Pere
Pere
  • 6.6k
  • 1
  • 18
  • 34

No, you can't interpret the overlapping confidence intervals this way.

First of all, you should keep in mind that the market share may have changed in those few months. You can even test if that change is significant - that is, whether the change is large enough to be detected with such a small sample.

If you want to assume that the market hasn't changed, you can use your boosted sample - that I assume to be the initial sample plus 600 new observations - and work with it. Then, the best information you can have is the confidence interval from this larger sample.

Anyway, I wouldn't be very sure that the market share hasn't changed and I would consider working only with the 600 new observations - specially if a test had rejected that the market share is the same.