2
$\begingroup$

It is my understanding that standard deviation does not work well as a measurement for distributions that are heavily skewed. If I have a heavily right-skewed distribution, should I simply use the CDF of an appropriate distribution to determine what percentage of values would lie beyond a certain point on the curve? What well-defined methods exist for determining the appropriate CDF given an empirically gathered distribution?

An example of the sort of 'rough' empirical distribution I'm talking about would be this:

enter image description here

$\endgroup$
2
$\begingroup$

You could always use the the (complement of) the empirical CDF (ECDF) to estimate proportions beyond any particular value.

http://en.wikipedia.org/wiki/Empirical_distribution_function

You can even get nonparametric intervals for such quantities.

Asymptotically, you could use:

$$\sqrt{n}\big(\hat F_n(t) - F(t)\big)\ \ \xrightarrow{d}\ \ \mathcal{N}\Big( 0, F(t)\big(1-F(t)\big) \Big),$$

to get an interval for the estimate. See the linked article for a number of other results that might be of some use to you.

$\endgroup$
  • $\begingroup$ Thanks. This is probably a silly question but what do you mean by nonparametric intervals here? $\endgroup$ – 114 Sep 30 '14 at 1:23
  • $\begingroup$ @114 I mean intervals that are distribution-free, which is to say intervals that are of the same algebraic form irrespective of the distribution* from which they were drawn. $\quad$ * -- assuming we have continuous distributions, at least; discrete ones won't actually be distribution free. As an example, distribution-free intervals might be backed out of goodness of fit tests using the ECDF such as the Kolmogorov-Smirnov, Cramer-von Mises or Anderson-Darling. The asymptotic result above is distribution-free, as are bounds given by some of the inequalities at the link in the answer ...(ctd) $\endgroup$ – Glen_b -Reinstate Monica Sep 30 '14 at 2:13
  • $\begingroup$ (ctd) ... there are also related distribution-free intervals for population quantiles - e.g. see here. There's also the bootstrap which can be used to generate intervals. ... One thing about your plot does concern me, though - it looks like your actual data may be discrete. $\endgroup$ – Glen_b -Reinstate Monica Sep 30 '14 at 2:21
  • $\begingroup$ That's correct, there are discrete data points that I would like to build a continuous distribution around. I believe I see now what you mean by distribution-free, thanks! $\endgroup$ – 114 Sep 30 '14 at 3:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.