4
$\begingroup$

Looking at Google Analytics, I can't help but think that the summary stats which present averages have the potential to be very misleading (and probably are, in practice). For some of these summary stats, there doesn't appear to be an easy way to determine the distribution of the numbers underlying the data. For other stats, there's apparently no way to determine the underlying distribution. It's clear enough that the underlying distributions for some of these stats are likely to be non-Gaussian.

So I'd like to know - if you have "average" summary stats without knowing the distribution of the underlying numbers, how do you interpret or derive meaning from them?

Improve this question
$\endgroup$
16
  • 1
    $\begingroup$ Possibly of interest: should the mean be used when the data are skewed?. $\endgroup$
    – Alexis
    Commented Mar 13, 2017 at 23:28
  • 1
    $\begingroup$ @Alexis, thanks -- I'd totally forgotten that thread (even though I posted an answer -- amusingly, I was reading an answer yesterday to which I thought "Gee I wish I knew as much about this" only to find it was my own answer). I think that post is probably sufficiently close to call a duplicate (in that answers to that question in effect respond to this question). If they're not sufficient for Dreadnaught's purposes, OP can either edit to distinguish (ie focus on what isn't answered there) and flag to reopen or post a new question aimed at whatever specific issues were not covered. ... ctd $\endgroup$
    – Glen_b
    Commented Mar 14, 2017 at 0:00
  • $\begingroup$ ctd .. This one and the one you linked to are the sort of really fundamental question about which volumes could be written but solid practical advice is very difficult to offer unless we go into a great deal of figuring out what we're really trying to find out and studying a great deal about the variables of interest. Good advice is especially difficult to offer for people not already having a fairly solid background in statistical ideas (where it's most needed, really, since the overwhelming majority of people for whom such a question is relevant don't have more than a smattering of it). $\endgroup$
    – Glen_b
    Commented Mar 14, 2017 at 0:00
  • $\begingroup$ There are some other good links in the sidebar (under "Related"). Of possibly some marginal relevance is this other question about when the mean is more efficient than the median with symmetric distributions. $\endgroup$
    – Glen_b
    Commented Mar 14, 2017 at 0:02
  • $\begingroup$ @Glen_b - Above you stated, "studying a great deal about the variables of interest..." The dilemma here is that my ability to study the variables in question is constrained by the lack of availability of the data. The issue is not one of non-normal distributions (maybe I should delete that part of the question?), or of the fundamental meaning of what a mean is. I'm clear on those concepts. Part of my intent is to ask whether interpreting certain descriptive stats is an exercise in self-deception when the ability to determine the shape of the data one is dealing with, is severely limited. $\endgroup$
    – Dreadnaught
    Commented Mar 14, 2017 at 1:00

1 Answer 1

Reset to default
4
$\begingroup$

As Gertrude Stein might have written had she been a statistician, "Do we suppose that all she means is that a mean is a mean is a mean"$^{\,*}$.

You don't have to have normality for the mean to be meaningful. I use it quite happily when I think of an exponential or a Poisson model, or a binomial, or with a discrete uniform, for example, and even when I don't really have a good model for the distribution. (It's not necessarily the only statistic I care about, though, but it's a handy thing in a wide range of situations.)

The sample mean (aside from any interest in its own right as a kind of data-summary) is an unbiased estimator of the population mean (when it exists - but that will probably apply to all or at least nearly all of the measures you're looking at in practice), and converges to it.

Two big considerations when trying to estimate population quantities:

  1. What quantities are of interest to you?

  2. What's a good way to estimate them?

If you you include "the mean" in 1. but don't know much about the distribution that you're sampling from, then you don't know a good way to estimate the mean (i.e. it's hard to say much about 2.); at least the sample mean has some useful properties, and should get there eventually, at least if the population you're sampling is the population of interest. In that case, you can still interpret the mean as, well, the mean, and as an estimate of the population mean.

Imagine, for example, I was sampling from a lognormal distribution (but didn't know that). The sample mean is going to "work" as an estimate of the population mean. [Though depending on how much skewness we're dealing with, might be quite noisy, and if we want to give an interval for the mean, it's worse.]

However, while the mean has a few nice properties when you're sampling from what you want to make inferences about, you're right to approach the mean with caution (it's not very robust, for example, so even a teeny bit of contamination is a problem for it, and that can certainly mislead us if we're interested in understanding something about the population absent the process of contamination$^\dagger$), but by the same token you needn't be overly focused on Gaussian distributions if you actually want to know about the population mean.

$*$ which if she'd written it might have been in a work called Operas and averages

$\dagger$ it might be better to accept a potentially substantial bias induced by a slight robustification than a potentially unlimited amount if the contamination is wild enough

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, Glen. I didn't mean to attribute excess importance to normality, in my question. My focus is more on the situation where you literally may not know what sort of distribution you're dealing with, which seems to be the case with a certain popular analytics tool. You did address that, and helped me clarify my thinking a bit about the topic. Thanks! I don't have enough rep to upvote, but I'll accept the answer in a couple of days, if there's no further input. $\endgroup$
    – Dreadnaught
    Commented Mar 13, 2017 at 23:21
  • $\begingroup$ OMG +1 for G. Stein humor!!! :) Brief aside: when I visited Père Lachaise Cemetère, where Stein and Toklas are interred, their monument was covered in stacks of rocks and some flowers, whereas a few avenues over, Oscar Wilde's tomb was gifted with candy and poems or other written missives. :) $\endgroup$
    – Alexis
    Commented Mar 14, 2017 at 0:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.