Comparison of means from different sample sizes

Question

I have a data set which contains avg profit and the number of samples for each data point, no more information.

I'd like to compare the data points and decide which one to focus on more. However, I'm not sure how to take the sample size into account. As a simplified example, I have written the python code below.

import matplotlib.pyplot as plt; plt.rcdefaults()
import numpy as np

objects = ('A', 'B', 'C', 'D')
y_pos = np.arange(len(objects))
sample_sizes = [10,5,20,15]
sample_avgProfit = [12,14,2,4]
weighted_mean = 0
for i in range (len(sample_sizes)):
    weighted_mean += sample_sizes[i]/sum(sample_sizes)*sample_avgProfit[i]

weighted_proportion = []
weighted_sum = sum([a*b for (a,b) in zip(sample_sizes,sample_avgProfit)])
for i in range (len(sample_sizes)):
    weighted_proportion.append(sample_sizes[i]*sample_avgProfit[i]/weighted_sum)


plt.bar(y_pos, sample_avgProfit, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.title('avg profit')
plt.show()

plt.bar(y_pos, weighted_proportion, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.title('weighted proportion')
plt.show()

You'll see that B has a higher avg profit, but since its sample size is small, when I consider weighted sum and average, then A shows a higher proportion weight (BTW, is this the right term for the values I calculated?)

So my question is:

1. Am I using the right metric to compare the data points?
2. How to interpret the results? in this example, product A might have a higher weighted proportion, but still B has a higher avg price. What is the right way to make a decision in this case?

Answer 1

Comparing means requires information about the variability of the estimates of the means. That is, in order to test hypotheses of the type "is the mean in group B higher than that of group A?" you'd need to know something about the standard deviation in each group. If one group varies wildly, then the mean is likely unstable too and should not be "trusted" as much as if the variance was very small. Without such information, you'd have to make heroic assumptions about what the standard deviations in each group is.

To me, the main question here would then be if the difference in the two groups' observed means is of practical importance to you? If the smaller group's mean is only slightly larger than the much larger group's mean, and the larger group's mean is good enough, then perhaps going with the group where you have a more precise estimate of its mean is the safer bet. But again, this is based on the assumption that the standard deviation in the group that you have more observations from is not so much larger group that it counteracts the larger sample size.