0
$\begingroup$

I have the following data set:

340
1680
1490
140
56
188
177
15
99
5
31
160
67
10
140
35
105
67
34
150
77
20
122
19
250
59
9
144
199

I calculated the arithmetic mean $\mu = 203.0344828$ and the standard deviation $\sigma = 385.1202663$.

As the standard deviation is higher than the mean, I am wondering whether it makes sense to use the standard deviation to indicate the spread of the data, since it seems to be highly sensitive to the values $1680$ and $1490$. If only the arithmetic mean and the standard deviation is given, would that be misleading to readers?

$\endgroup$
  • $\begingroup$ About notation: Symbols $\mu$ and $\sigma$ are usually reserved for population mean and SD. Usual notations for sample mean and SD are $\bar X$ and $S.$ $\endgroup$ – BruceET Aug 2 '18 at 20:21
4
$\begingroup$

I agree with @ERT (+1) that just giving the sample mean and standard deviation might give a false or confusing impression of your data, and that some graphical presentation that shows the two outliers (such as a histogram, with or without the smoothing density estimator) the would be useful. My purpose here is to discuss additional possibilities.

Without knowing the source of your data or the audience for your summary of them, it is difficult to give advice about what summary would be meaningful.

Here is a listing of your $n = 29$ observations, sorted from smallest to largest:

sort(x)
 [1]    5    9   10   15   19   20   31   34   35   56
[11]   59   67   67   77   99  105  122  140  140  144
[21]  150  160  177  188  199  250  340 1490 1680

If limited to a brief numerical summary, I would say the 29 observations run from a minimum of 5 to a maximum of 1680, with a sample median of 99. Also, that all but two observations are 340 or smaller. If your listing puts data in time order of collection, it may be worthwhile to mention that the two extreme observations 1490 and 1680 occurred near the start (the second and third observations).

median(x)
[1] 99

If the audience for your summary is familiar with statistical terminology it may be useful to give the summary statistics provided by the function summary in R:

summary(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
      5      34      99     203     160    1680 

Then a boxplot might also be useful (but only if you wouldn't need to take half a page to explain what a boxplot is).

boxplot(x, horizontal=T, col="skyblue2", pch=19)

enter image description here

In any case, if you have a any clue as to the possible reason for the two extreme outliers, you might include a possible explanation.

For statistically more sophisticated audiences, it might be useful to show the nonparametric 95% confidence interval $(72, 152)$ for the population median that can be found in R using the Wilcoxon signed-rank method wilcox.test(x, conf.int=T, conf.lev=.95). Or to provide a 95% nonparametric bootstrap confidence interval for the population mean (which I did not compute).

$\endgroup$
  • 1
    $\begingroup$ +1 for the box plot, that could be insightful for the reader as well. Plus, it takes up less space on the page than a related histogram / density. The only problem with that plot: it is more difficult to visualize the density. $\endgroup$ – ERT Aug 1 '18 at 18:23
  • $\begingroup$ For what it may be worth: 95% nonparametric bootstrap CI for $\mu$ (treated as scale parameter) is $(114, 470),$ which contains sample mean $\bar X = 203.$ Bootstrap CI for pop median $\eta$ is $(56, 155),$ which contains sample median 99 and is somewhat similar to Wilcoxon CI. [However, bootstrap of median is based on only 20 uniquely different resampled medians out of 100,000, with many repetitions of some resampled median values.] $\endgroup$ – BruceET Aug 2 '18 at 20:38
1
$\begingroup$

In short, that would be misleading for your readers. Let's look at a quick R example:

a <- c(340, 1680, 1490, 140, 56, 188, 
       177, 15, 99, 5, 31, 160, 67, 10, 
       140, 35, 105, 67, 34, 150, 77, 20, 
       122, 19, 250, 59, 9, 144, 199)
hist(a, prob=TRUE, breaks=15)
lines(density(a), col="blue", lwd=3)

From this data, we come out with the following plot:

Histogram of your data, with empirical density

Clearly, your distribution has a very heavy tail. Most of your data is clustered around your mean, with a few key outliers. A better way to communicate the important aspects of this fairly limited dataset would be to show the viewers a histogram, or at least explain the heavy-tailedness of your dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.