0
$\begingroup$

I have a some datasets that I want to take an average of - a typical dataset might look something like this:

0
0
0
1
100
100
10000
30
10
10
0
0
0
0
0

Ordinarily I would use the geometric mean for something like this - but unfortunately I have a lot of zeros in there. I don't want to use the median because the truly extreme values (~10,000) are quite rare and I want them to be included when they do appear.

So I was wondering if a valid approach might be to first take the geometric mean of the nonzero values, then take the weighted arithmetic mean of the geometric mean together with the zero values.

So for the above dataset:

Geometric mean of nonzero values = Geom(1,100,100,10000,30,10,10) = 43.6

Weighted arithmetic mean = (7*43.6 + 8*0)/15 = 19.075.

I think this seems better than the simple arithmetic mean, which returns 683, and better than the median, which returns 0.

Is there anything wrong with doing this, and if so could anyone suggest an alternative. Any advice on how to calculate an error for this would also be appreciated.

EDIT: I realised that my plan clearly doesn't work, as if I had a dataset consisting of 10000 from the above dataset but all other values replaced with 0, I would still end up with a mean of 666 - which is a much higher number than 19.075, despite the actual data going into it being lower. So now I have no idea how to approach this problem, and would appreciate any insight!

$\endgroup$
  • 1
    $\begingroup$ What use or interpretation do you want to put on any single-number summary? Why do you think such a summary is needed? $\endgroup$ – Nick Cox Jan 30 '18 at 1:21
  • $\begingroup$ My data comes from seed count estimations - how many seeds are released from a tree per minute, which can vary a lot. I want to take an average seed count for each 20 minute stretch. I want to model the relationship between seeds released and environmental variables such as turbulence and temperature. In order to calculate turbulence you need to take a 10-20 minute average of wind data, so that is partly why I need to take a 20 minute average of the seeds. Also, my data is just really noisy and quite autocorrelated, so that's another reason why I want to take an average. $\endgroup$ – agorapotatoes Jan 30 '18 at 2:40
  • $\begingroup$ You might want to just avoid the whole problem by using zero inflated models. In R, zeroinfl in the pscl package and dzim and zim in the ZIM package. $\endgroup$ – G. Grothendieck Jan 30 '18 at 5:28
  • 1
    $\begingroup$ I agree with @G.Grothendieck Think about modelling the distribution, not identifying a single reduction. (Fraction of zeros and mean and geometric mean of the rest might be a better reduction.) $\endgroup$ – Nick Cox Jan 30 '18 at 7:43
  • $\begingroup$ See this answer stats.stackexchange.com/questions/234816/… $\endgroup$ – Brash Equilibrium Apr 17 '18 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.