Given two sets of data, what could explain similar means but different standard deviations? Given two sets of data of user activity, both of which appear to be in an exponential distribution, I have calculated the mean and standard deviations using both a mean/deviation and a sample mean/deviation (sample size = 30, number of samples = 10k):

A (size: 627,000):

*

*Raw --  μ = 45.947, σ = 114.2, σ/√n = 0.14422 
B (size:3570):

*

*Raw --  μ = 46.43, σ = 116.1 
Using the above data, it seems that the two means differ by a statistically significant amount, and thus allow us to say with confidence that the average for B is greater than the average of A.

A

*

*Sampling --  μ = 46.174, σ = 21.256 
B

*

*Sampling --  μ = 46.786, σ = 21.366 
Using the sampling data standard deviations, we see that the difference in means (0.612) is much less than the deviation, it seems that the means do not differ by a statistically significant amount.

So given the above, which is right? Can we say that these data sets differ? If the underlying distribution is exponential, are the above tests even accurate?
 A: You have a huge sample size.  Statistical significance means that the probability of an observed difference that was observed or a more extreme difference is less than the designated significance level UNDER THE NULL HYPOTHESIS that the MEANS ARE EXACTLY EQUAL.
Consequently if you have a very large sample size very small differences can be detected.  To see this consider the standard error of the estimate.  It is reduced from the population standard deviation by a factor of the square root of the sample.  What you are confusing here is the two concepts "statistical significance" and "practical significance."  For sample sizes of 627,000 and 3,570 you have an estimate of the mean difference that is accurate enough to statistically detect a difference as small as 0.6 (observed estimate 0.612) but as you see both means are around 46 and so this difference is not "practically" significant.
In such cases the practical significance is what should be important to you.  Recognize that two populations will never have exactly identical distributions and hence identical means and so even small unimportant differences can be detected with large enough sample sizes.
A: Actual the previously proposed answer does not quite answer the question as to why one would more likely see aberrations in the estimated variances between the two sets. The precise answer is that the sample variance and its associated standard deviation is generally speaking a more noisy statistic for distributions with large kurtosis. That is, the sample variance, and the standard deviation derived therefrom, could have a relatively larger variance than the sample mean depending on the distribution.
In particular, per Wikipedia on the variance of the sample variance:
$$
{\rm Var}(s^2) = ({2/(n-1)+ kurtosis/n})* sigma^4
$$
and per an approximation formula to compute the variance of the standard deviation (also discussed in the same Wikipedia reference) implies a value that would exceed the variance of the sample mean, $sigma^2$/n, when, per my calculations, the kurtosis is greater than 2. This would confirm the usual observation for exponential data of an increased in noise for the sample variance and standard deviation relative to the sample mean (which has special meaning in the case of exponential distributed data, as in the latter the sample standard deviation is an alternative nonlinear estimator of the mean as well).
[EDIT] A interesting thought would be to examine a combined estimator with each estimate receiving weights, adding to one, based on the reciprocal of their respective expected variances. Such a weighting scheme, while different in incorporating a nonlinear and linear estimators, is somewhat related to an estimator based on a  weighted average of order statistics, where the respective covariances of the order statistics are also included in the calculation, and may offer some benefit.
I have since constructed a simulation spreadsheet of 100 groups of 11, for a total of 1,100 exponential deviates generated. Here is a somewhat typical result:
Expected Mean: 2.000  Observed mean: 2.046  Observed Variance/Expected Variance of the group means for 100 iterations: .344/.364  Average of all the groups Standard Deviations: 1.961 Expected: 2.000  Observed variance of all the group standard deviations: .684  My computed expected variance of the group standard deviation per the approximation formula: .746 (I did observe an evident suggested high bias on this statistic)  Average of Group variances: 4.523 Expected 4.000  Observed variances of the group's variances: 15.46 (observed to be very noisy) and finally, the expected value for the variance of the group variances per formula above: 11.93.
