2
$\begingroup$

At first, I want to say, I know that mean is outlier sensitive.

Problem

I have to talk about the lifestyle of the students, where the data is quantitative. Let's say I have to talk, how much time a student spend in the classroom (A), how much time a student spend in library (B) etc. Some of the data set (e.g. A) contains outlier. Since I believe that extreme users are representative of the population who does/uses extremely, I do not want to remove outliers and I want to write about the lifestyle of the students using the mean value without removal of outliers.

What creates the problem?

This paper says mean is outlier sensitive and many other articles says, you should remove outliers when calculating mean. However, I have never read a research paper where researchers have talked using median value, at time of talking lifestyle. For instance: I have never read a paper where researchers said "students spend 1 hour in the library" where 1 hour was median value.

My Question

I will show median values for each type of data (e.g. spending time in library, spending time in classroom) along with mean value in the table. Without removal of outliers, will it reasonable if I talk about the lifestyle of the students in everywhere except the table using only mean value?

Thanks in advance!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

Removal of outliers for computing the mean is certainly not mandatory, and may actually in some cases not make much of a difference (if the outliers are not that far out, and their percentage is small). Surely removal of outliers is more sensible if there are reasons to believe that these observations are erroneous, but you seem to be sure that yours are not, which is a good reason not to remove them.

If in fact the mean is strongly influenced by the outliers, I think it's better to comment using more than one number to get the whole picture. Using mean and median is one option, but maybe in your case it is even more sensible to say something like "the mean is 3 hours, in fact the vast majority is below 3.5 hours, but some 5% spend more than 7 hours per day XXX".

Improve this answer
$\endgroup$
5
  • $\begingroup$ Great answer, thanks for your information "the mean is 3 hours, in fact the vast majority is below 3.5 hours, but some 5% spend more than 7 hours per day XXX". $\endgroup$
    – Md Sabbir Ahmed
    Oct 9, 2019 at 15:07
  • $\begingroup$ Can you please cite a paper or reliable article where you got "Surely removal of outliers is more sensible if there are reasons to believe that these observations are erroneous". I guess, this is a belief. However, providing paper would help me more to go on. $\endgroup$
    – Md Sabbir Ahmed
    Oct 9, 2019 at 15:09
  • $\begingroup$ I will accept your answer if you please can provide a research paper or reliable article regarding this. $\endgroup$
    – Md Sabbir Ahmed
    Oct 9, 2019 at 15:11
  • $\begingroup$ I'd expect that this can be found somewhere, but my answer is not taken from some published paper but rather a result of experience and thought, (critical) reading of lots of literature about robust statistics and outliers, and regular attendance of conferences where these things are discussed. Unfortunately I'm not that familiar with the literature that explains these things properly to beginners. In order to find something published that says what I've just written I'd need to google and you can do that yourself. $\endgroup$
    – Christian Hennig
    Oct 9, 2019 at 15:27
  • $\begingroup$ Some of this is just simply logical, like that outliers should be removed if they are erroneous (although in many cases one can't know for sure) and that if they are not erroneous you'd remove valid information, which can do harm. About the influence of outliers you can find out yourself, comparing your mean before and after outlier removal. Sorry I'm more of a self-made statistician than a literature recommender. $\endgroup$
    – Christian Hennig
    Oct 9, 2019 at 15:32
2
$\begingroup$

First off, kudos on recognising that outliers should be retained!

You seem to be most interested in how to characterize the population (students). Means and medians are both commonly used to convey what a 'typical value' of the population (loosely speaking) is. If the distribution is highly skewed and the mean does not capture a 'typical value' well, you may not want to use it (though there are exceptions to this). In this case, it's quite reasonable to use the median instead. Alternatively, you can use the mean along with confidence intervals (which can be asymmetric and convey a skewed distribution). Another common strategy that may be less useful to you in this specific case is to transform (e.g. take the log) the data so that it becomes less skewed; this is frequently a good way to plot data that is highly skewed.

Bottom line: it depends on what you want to communicate. In your case, I would use the median, but be sure to make this clear so your audience understands what is being discussed.

Improve this answer
$\endgroup$
2
  • $\begingroup$ we have dataset where each category contains a specific type of data. So, if we start talking about mean, for the category which is highly skewed can not be described using median value, I think. I think, we have to describe the lifestyle using a specific method mean/median. $\endgroup$
    – Md Sabbir Ahmed
    Oct 9, 2019 at 15:01
  • 1
    $\begingroup$ @SabbirAhmed You can use medians for all categories to make it simple. To oversimplify a bit, the more symmetric a distribution is, the more similar the mean and median are. For a perfectly symmetric distribution, the mean and median are identical. So there's no downside to using the median for less skewed distributions too. $\endgroup$
    – mkt
    Oct 9, 2019 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.