Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am working on a study to quantify average working hours for doctors. However, when I leave it empty for respondents to fill up, it remains unfilled.
Changing it into categories as above yield better responses. (categories of working hours; 1 to 10, 11 to 20, 21 to 30)
Now is it possible for me to derive a mean from these categorical data?
n1 x midpoint of category 1 + n2 x midpoint category 2 ..... divide by total n
It's somewhat misleading to just lump this in with ordinal data; I'd called it "binned data" though formally it's interval-censored data (and there are a variety of other terms that might be used).
You can certainly talk about the population mean (since the underlying scale really does have a mean) and how to estimate it, bringing in what is understood about the underlying variable to help figure out ways to estimate it well from the bin-counts and bin-boundaries.
While it's common to use the mid-point in such cases it's not always the best possible option. However, one can get some idea of how biased that might be under some set of assumptions so it's possible to get a sense of whether it really matters all that much.
Where the underlying density is decreasing, the correct "midpoint" to use would be left of half way, and if the underlying density is increasing, the correct "midpoint" to use would be right of half way.
If you can come up with a plausible distributional model for the underlying variable, the mean can be estimated from the binned data via maximum likelihood (for example).
Even in the absence of any model at all, one can place limits on the mean, since the lowest the mean can be is when all the values are at the low end of each interval and highest when they're all up at the high end of each interval. [Even if the upper category is seemingly open-ended, there's still likely an effective upper bound on hours worked. e.g. it's simply impossible to work 25 hours in a day or 169 hours in a week, even if you never need to eat or sleep. Likely there's some other substantially lower bound beyond which nobody can go for one reason or another.]
No, I would not consider that to be valid. The problem is that the mean of the true values in each category is not likely to be the midpoint. For example, there are probably many more people who would answer 10 hours than one hour - so the average hours worked will be more than 5.5, but you are assuming that the mean is 5.5. Hence your estimate will be biased.
What you could do is consider it to be a scale with a weird non-linear transformation - saying something like "On a scale where 1 = 1 to 10, 2 = 11-20 ... the mean score was 1.8."
But if you only have three categories, you can just say "22% of people worked 1-10 hours, 43% worked 11-20 hours ..." Unless there's a very good reason that you need a mean, I would do that.
Possible? Yes, as you've shown.
Valid? Depends on what you mean. It's an estimate, and estimates can be biased.
Consider the case where half the respondents give you exact measurements (e.g. 22 hours) and then half give you a binned estimate (e.g. 21-30 hours). If you calculate the average binned estimate as you showed above
n1 x midpoint of category 1 + n2 x midpoint category 2 ..... divide by total n
then you could add that number with the mean exact measurement, divide by 2, and get an estimate of the average working hours.
Or maybe you want to give more weight to the mean exact measurement, and so you could do a weighted average of the two means to estimate the average working hours.
A third estimator could look like this: Bin the exact measurements into the three categories, and then find the deviation of the empirical average within a bin from the midpoint of that bin. (e.g. with exact hours observed as 22, 24, and 23, the average within the bin is 23, which deviates from 25.5 by 2.5). Then, you may choose to use the empirical average within the bin (instead of the midpoint of the bin) in order to calculate the average work hours from the observations that had measurement in categories/bins:
n1 x empirical average (from observations with exact measurements) within bin 1 + n2 x empirical average within bin 1 ..... divide by total n
Another estimator could take a parametric assumption and/or Bayesian framework to estimate the average from the obervations with binned measurements.
There are plenty of estimators. Theory of statistics can show that some may "work better" than others. If you're a frequentist you'll probably want one with 95% asymptotic coverage. Those estimators would probably be the "most valid".
As another answer points out, your proposed method is likely to be biased, and so maybe not as "valid" as you would like. Reporting the percent of observations in each bin, however, is a very good way of explaining your data. if you feel strongly above giving an estimate of the overall mean, you could do so, but be sure to be clear that you used a midpoint-calculation statistic like your proposed method, and perhaps state that your estimate is not very precise.
