Does it make sense to use deciles for distribution that are skewed. For example, consider the exponential distribution with lambda >= 1.5 or lambda distribution with k = 1 and theta = 2 or any other that is highly overweight. Or for cases like this, are there better/smarter ways to segment the data?

Thanks for any insight.

  • $\begingroup$ Not sure whether you're talking about deciles of the population or of the sample, nor your purpose in 'segmenting'. By 'overweight', do you mean 'right skewed'? // Generally speaking, quartiles and deciles are often used to describe distributions and data. $\endgroup$ – BruceET Jun 24 '19 at 4:41
  • $\begingroup$ Welcome to the site. Your question needs a lot more detail to be answerable. My post (how to ask a statistics question)[statisticalanalysisconsulting.com/… may help you formulate a question that can be answered. $\endgroup$ – Peter Flom - Reinstate Monica Jun 24 '19 at 10:47

Consider the following random sample of 1000 observations from an exponential distribution with rate 2 (mean 1/2). Various statistical software programs use slightly different rules for determining quantiles of samples, I used R here. (For samples as large as 1000, differences in quantile rules are usually not if practical importance.)

x = round(rexp(1000, 2), 3)

A boxplot illustrates the right-skewness of this distribution. The many 'outliers' are characteristic of exponential samples.

enter image description here

Here are the deciles of the sample:

q = quantile(x, seq(0, 1, by=.1));  q
    0%    10%    20%    30%    40%    50%    60%    70%    80%    90%   100% 
0.0000 0.0529 0.1108 0.1767 0.2480 0.3225 0.4208 0.5476 0.7280 1.1684 3.5480 

Because of the right skewness of this distribution, the deciles are increasingly far apart as observed values increase.

Ordinarily, it is not a good idea to make histograms with unequal bin widths, but to illustrate this pattern of deciles, I used them as cutpoints in binning a histogram of this sample. Each bar of the histogram represents about 100 observations. The red curve is the density function of the exponential population.

enter image description here

Here is R code for making the histogram.

hist(x, prob=T, br=q, col="skyblue2")
  curve(dexp(x, 2), add=T, col="red", n = 10001)

Here are the deciles of the distribution $\mathsf{Exp}(\text{rate}=2).$

round(qexp(seq(0,1, by=.1), 2), 4)
 [1] 0.0000 0.0527 0.1116 0.1783 0.2554 0.3466
 [7] 0.4581 0.6020 0.8047 1.1513    Inf

Except for the maximum, the sample deciles estimate the population deciles.

  • 1
    $\begingroup$ Deciles of a sample of the population. Let's say we want to classify shopper types. One way to do it would be to use deciles to determine who is an infrequent shopper, .... up power shopper or however you want to call them. For highly skewed data, are deciles a good method for determining segments or should another method be used? $\endgroup$ – dustin Jun 24 '19 at 5:14
  • $\begingroup$ That seems to be a reasonable application. $\endgroup$ – BruceET Jun 24 '19 at 5:15
  • $\begingroup$ Are deciles appropriate for distribution types? Should they log normalized if skewed? Are there any standard procedures or write on this? I couldn't find anything while I was searching online. $\endgroup$ – dustin Jun 24 '19 at 5:48
  • $\begingroup$ I see nothing wrong with using sample deciles to describe skewed distributions. Is there some specific difficulty you expect? $\endgroup$ – BruceET Jun 24 '19 at 5:51
  • $\begingroup$ I am just trying to understand the common practices for when or when not use deciles. Many methods can be applied incorrectly so I am trying to avoid that $\endgroup$ – dustin Jun 24 '19 at 5:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.