2
$\begingroup$

I have a small number of samples (5) of a large population (~10,000). The samples are percentages and hence I know from the context that no answers are possible below 0% or above 100%. From this one set of samples I've calculated my t statistic and confidence intervals; however, for anything above 95% confidence my interval width is such that the predicted upper limit is above 100.

My understanding is that is you have specific knowledge like this you can create a skewed Student's t distribution. First, is this correct? Second, how would this apply to the upper and lower limits of my confidence interval? I understand that two values are important in calculating this, the skewness and kurtosis, but I'm not sure how to use them.

My skewness value: -1.82

My kurtosis value: 3.448

Sample Values:

79.465
91.905
91.096
88.144
90.599
Improve this question
$\endgroup$
4
  • $\begingroup$ Very unclear and confused. Show us your data and what you did with them e.g. in terms of some program code. You have skewness and kurtosis of data all mixed up with the properties of sampling distributions of test statistics. It is possible to generalise Student's t distribution to skewed variants but that seems way beyond rather simple tests for comparing means that appear to be your focus here. $\endgroup$
    – Nick Cox
    Apr 18, 2014 at 12:23
  • $\begingroup$ I'm not clear on what has confused you here. My data is 5 samples, I've edited them into the question now, not that I'm sure how much this can help you with the general method that I'm asking for. I've determined I require I student t distribution based on the unknown population characteristics. I've also determined that the distribution will be skewed and potentially no centred. I'm interested in how I might calculate the upper and lower limits based on a given confidence value, when taking into account the absolute maximum value is 100 (minimum is 0). $\endgroup$
    – FraserOfSmeg
    Apr 18, 2014 at 12:49
  • $\begingroup$ Just by looking at your sample, I can say that you can't compute 95% confidence reliably. Your sample size is 5 and the population is 10,000. The variance is 26. Intuitively it's way too large for the relative size of your sample given that the support is from 0 to 100. $\endgroup$
    – Aksakal
    Apr 18, 2014 at 13:12
  • $\begingroup$ I did not say I was confused. If each percent is itself a summary of a sample, then you need the original data. Your summaries do not appear highly skewed to me. Are you measuring proportions or reporting the proportions of a binary variable? $\endgroup$
    – Nick Cox
    Apr 18, 2014 at 13:28

1 Answer 1

Reset to default
2
$\begingroup$

Here's a simple Monte Carlo simulation. Assuming that the population distribution is Beta with parameters 80 and 1.1, I draw samples of 5 observations, then plot the histogram of their mean.

x=mean(100*betarnd(80,1.1,5,100000));
subplot(2,1,1)
plot([94:0.01:100],betapdf([94:0.01:100]/100,80,1.1))
title 'Population distribution'
subplot(2,1,2)
histfit(x,40,'t')
title 'Sample mean'

enter image description here

You can see that the sample mean does not fit neither normal nor t-distribution, so t-statistics would be useless. I don't think that you can derive the distribution of the sample mean without knowing something about the distribution of the population beyond its bounds. Your sample variance is 26, i.e. too high relative to the size of the population and its bounds in order to apply CLT here or other asymptotic analysis reliably.

Now if instead of assuming some random population distribution, let's fit Beta distribution to your data.

xp=[79.465
91.905
91.096
88.144
90.599]

% fit Beta distribution to the sample
ab=betafit(xp/100)
% generate many samples of 5 observations
x=mean(100*betarnd(ab(1),ab(2),5,100000));

subplot(2,1,1)
plot([0:0.01:100],betapdf([0:0.01:100]/100,ab(1),ab(2)))
title 'Population PDF'
subplot(2,1,2)
histfit(x,40,'tLocationScale')
title 'Sample mean distribution'

phat=mle(x,'distribution','tlocationscale')
pd = makedist('tLocationScale','mu',phat(1),'sigma',phat(2),'nu',phat(3))
disp 'Conf. intervals of population mean'
pd.icdf([0.025 0.975])

This gives an output:

pd = 

  tLocationScaleDistribution

  t Location-Scale distribution
       mu = 88.2223
    sigma = 1.82489
       nu = 89.9698

Conf. intervals of population mean

ans =

   84.5968   91.8478

enter image description here

The 95% confidence interval seems reasonable, but the t Distribution fit is not so good, it's not skewed as the sample mean distribution. You could do something along this line, so long that you have a good idea about the population distribution. I still don't tink that the confidence interval would be reliable given the tiny size of the sample and its high variance.

Improve this answer
$\endgroup$
5
  • $\begingroup$ These are good observations. Would you be able to derive relevant and useful advice from them? Based on what you have shown, how could one proceed to develop a confidence interval for the mean of these data that is contained within the limits $[0, 100]$? $\endgroup$
    – whuber ♦
    Apr 18, 2014 at 13:55
  • $\begingroup$ @whuber, as you know Beta distribution can have very different shapes confined in [0,1] interval. I don't think there could be useful general result for the sample mean distribution solely based on the bounds [0,100] given that the sample is 0.05% of the population. More assumptions are necessary. $\endgroup$
    – Aksakal
    Apr 18, 2014 at 14:06
  • $\begingroup$ Thank you for this very complete answer. Stats isn't my field so I have a few clarification questions is you don't mind. First, doesn't generating many samples of 5 observations risk over-fitting? The mean shouldn't change significantly but wouldn't values like the standard dev be effected? Or does this method negate that? $\endgroup$
    – FraserOfSmeg
    Apr 18, 2014 at 23:22
  • $\begingroup$ Also, I've looked into Chebyshev's inequality and I thought this could be used for real world data to find the worst case scenario confidence. But this still gives me values above the expected 100. Might I be using this incorrectly? $\endgroup$
    – FraserOfSmeg
    Apr 18, 2014 at 23:43
  • $\begingroup$ Chebyshev's inequality is usually too obtuse to render useful results in practice. @FraserOfSmeg, you have to understand that there's a distribution of your observations, and there's a different distribution of the mean of a 5-observation sample. They're obviously related. When a sample is large, it is often possible to come up with the distribution of the sample mean without knowing much about the distribution of the observations. I'm afraid yours is not such a case. $\endgroup$
    – Aksakal
    Apr 19, 2014 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.