0
$\begingroup$

My scenario is the following:

I have a sample set of 1000+ rows, of which my variable of interest is a non-normal distribution. I haven't tested throughtly to check what kind of distribution it is, but I'm assuming a log-normal distribution.

As I have enough records in my sample, I was thinking on using what I have directly to calculate the mean, standard deviation and standard error. My goal is to capture the mean and the confidence interval.

Knowing that my distribution is not normal, do I have to sample it and 'approximate' to normal creating a distribution of the sample means?

Or can I use it directly and fetch the numbers I need from it?

Sorry if this is too basic, but I'm struggling to wrap my head around it.

$\endgroup$
2
  • 1
    $\begingroup$ What exactly is your goal? To obtain a point and interval estimate of the mean? The CLT states that this will be normally distributed (if the distribution is not too ugly and so on), so I don't see a need for what you are describing (basically a bootstrap procedure). $\endgroup$ Commented Nov 19, 2020 at 7:42
  • $\begingroup$ yes, you got it right! my goal is to fetch a mean and the confidence interval from the distribution. Could you please explain a bit more the part "The CLT states that this will be normally distributed"? The distribution is not too ugly. Just very right skewed $\endgroup$
    – Draconar
    Commented Nov 19, 2020 at 8:17

1 Answer 1

1
$\begingroup$

I assume it makes sense to model your observations as i.i.d. (identically and independently distributed) - there should not be obvious reasons in the way the observations were collected and measured to think that the observations are dependent (e.g., have a time series structure) or come in groups etc.

If this is so, the Central Limit Theorem says that normal distribution theory to do basic inference such as confidence intervals can be used for a large enough number of observations. How large is large enough depends on the distribution. Basically the only threat to approximate normality if you have 1000+ observations are gross outliers, either really extreme or many of them. If you don't have such a thing, normality based inference will be valid, and you will not need to simulate.

Note however that if your distribution is indeed lognormal or skewed in a way that a log-transformation brings it closer to normality, you may be more efficient and precise transforming the data to logarithms and run inference on the transformed data. The CLT means that (in all likelihood) normal inference is also valid for untransformed data, but it being valid doesn't necessarily mean that it cannot be improved.

$\endgroup$
1
  • 1
    $\begingroup$ I love you. This explained a lot of questions I had. thank you very much! BTW, my distribution fits in a Gamma distribution, as per R tools for distribution fitting. $\endgroup$
    – Draconar
    Commented Nov 19, 2020 at 10:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.