1
$\begingroup$

Let's say that we have one area and in this area there are designated 5 points, $a, b, c, d, e.$ For these 5 points we count the number of a specific feature. We capture the values for two different days, for example:

$\begin{array}[t]{llllll|l} \text{date} & a & b & c & d & e & \text{sum}\\\hline \text{first day}& 10 & 20 & 30 & 50 & - & 110\\ \text{second day} & 30 & - & - & - & 30 & 60 \end{array}$

The dashes mean that we have missing data for these points. We want to find the average of the number of features for these two days.

We know that when there is missing data, it is quite unlikely that the value is zero. So, the mean $ \frac {110 + 60}{2} = 85$ would lead to not a so correct result.

I can think of two approaches for this.

a) We can take the average for each point for both of the days; if for a day there is no data, we can consider that the value is the same, i.e. for point $a: \frac{10 + 30}{2}= 20$, for point $b: \frac{20 + 20}{2} = 20, \ldots$ and then we can add all the values. So, we will end up with $$ 20 + 20 + 30 + 50 + 30 = 150.$$

b) This approach I prefer more (but I am not sure about its correctness or if I am missing some crucial points) is to get a weighted average. So, because the first day we have more data, it seems more logical to me to give a larger weight to the sum of that date that we have more data, i.e. the first day.

So, we could say:

$$\frac{4\cdot(10 + 20 + 30 + 50) + 2\cdot (30 + 30 )} {4+2} = 93.33,$$ which means we don't take into account that much the second day, due to the large number of missing data.

I would like to know what measure is considered to be more appropriate in this case, in terms of which method is more likely to be more accurate if it makes sense at all.

If there are more elegant methods to deal with such situations, you are more than welcome to suggest them.

$\endgroup$
1
  • $\begingroup$ Hopefully you have more data than just these six values. $\endgroup$
    – AdamO
    Jan 9 at 20:17

2 Answers 2

1
$\begingroup$

I would considera two different approaches. a) Since you know that the first Point was 3 times more than the first day, you can Say that the same thing happened with the other points, in this case I assume that every point increase the feature independently.

b) here I consider that there is certain relation between the increasing between points of the same day. It means, if un the first day the feature was two times the feature of the first day than thee second day. You can Say that in the second day happened something similar, i.e 60 .

However, I think that it depends about what are you measuring, what represent each point, why the data was missing.

$\endgroup$
1
  • 1
    $\begingroup$ What you are advocating, essentially, is a model-based approach. This is different from what the OP has suggested. More could be done to expand and explain your approach. For instance, not only would you infer that Day 2 is 3 times that of Day 1, but that Slot B is 2 times that of Slot A. $\endgroup$
    – AdamO
    Jan 10 at 17:19
0
$\begingroup$

If there are missing data, you are using missing data methodology. You have to be explicit about what you're doing even if just ignoring them.

Clearly option B is completely biased. You have imputed 0 for the unobserved letters (bad) then taken a frequency weighted average of the a-e sum (good). Instead, the average for first day is 27.5, and 5 times that is 137.5. Similarly for second day: 150. (4*137.4 + 2*150 )/6 = 141.6 much closer to option A.

Option A is mean imputation which is fine for estimation. However, you do not use any frequency weighting here. In fact if you did, you'd get the same answer as B. (2*20 + 1*20 + 1* 30 + 1*50 + 1*30)/6 *5 = 141.6 again.

So they are not options at all, just row-wise vs column-wise approaches to calculating the sum.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.