How do you interpret an arithmetic mean without knowing the underlying distribution? Looking at Google Analytics, I can't help but think that the summary stats which present averages have the potential to be very misleading (and probably are, in practice). For some of these summary stats, there doesn't appear to be an easy way to determine the distribution of the numbers underlying the data. For other stats, there's apparently no way to determine the underlying distribution. It's clear enough that the underlying distributions for some of these stats are likely to be non-Gaussian.
So I'd like to know - if you have "average" summary stats without knowing the distribution of the underlying numbers, how do you interpret or derive meaning from them?
 A: As Gertrude Stein might have written had she been a statistician, "Do we suppose that all she means is that a mean is a mean is a mean"$^{\,*}$.
You don't have to have normality for the mean to be meaningful. I use it quite happily when I think of an exponential or a Poisson model, or a binomial, or with a discrete uniform, for example, and even when I don't really have a good model for the distribution. (It's not necessarily the only statistic I care about, though, but it's a handy thing in a wide range of situations.)
The sample mean (aside from any interest in its own right as a kind of data-summary) is an unbiased estimator of the population mean (when it exists - but that will probably apply to all or at least nearly all of the measures you're looking at in practice), and converges to it.
Two big considerations when trying to estimate population quantities:


*

*What quantities are of interest to you? 

*What's a good way to estimate them? 
If you you include "the mean" in 1. but don't know much about the distribution that you're sampling from, then you don't know a good way to estimate the mean (i.e. it's hard to say much about 2.); at least the sample mean has some useful properties, and should get there eventually, at least if the population you're sampling is the population of interest. In that case, you can still interpret the mean as, well, the mean, and as an estimate of the population mean.
Imagine, for example, I was sampling from a lognormal distribution (but didn't know that). The sample mean is going to "work" as an estimate of the population mean. [Though depending on how much skewness we're dealing with, might be quite noisy, and if we want to give an interval for the mean, it's worse.]
However, while the mean has a few nice properties when you're sampling from what you want to make inferences about, you're right to approach the mean with caution (it's not very robust, for example, so even a teeny bit of contamination is a problem for it, and that can certainly mislead us if we're interested in understanding something about the population absent the process of contamination$^\dagger$), but by the same token you needn't be overly focused on Gaussian distributions if you actually want to know about the population mean.

$*$ which if she'd written it might have been in a work called Operas and averages
$\dagger$ it might be better to accept a potentially substantial bias induced by a slight robustification than a potentially unlimited amount if the contamination is wild enough
