I have a range of readings from a test (for example:0.796, 0.109, 0.11, 0.11, 0.109, 0.109, 0.109, 1.78). I want to remove the extraneous values of these so that I get a more realistic view on the more accurate test result range. If I take the average (1.78) or the median (1.09) it doesn't give me that picture. I am not able to figure out how I could mathematically have a way to achieve this, because I don't want these extraneous values to skew my results.
4
-
$\begingroup$ Can you define what you mean by "extraneous"? Do you think that some of the readings represent a population other than the primary one & that you are not interested in? Do you think that some measurements have error in some way? Nb, in general, the median is considered the most robust measure of central location possible, so if the median isn't good enough, I'm not sure what would be. $\endgroup$– gung - Reinstate MonicaCommented May 28, 2013 at 14:05
-
$\begingroup$ Note that while ordinary English usage seems to be shifting recently to regarding skew and bias as synonyms, that's not standard in statistics, which has two terms for two different ideas. Skewness is asymmetry of distributions and bias is getting the wrong answer on average. I think you mean bias here. $\endgroup$– Nick CoxCommented May 28, 2013 at 14:11
-
$\begingroup$ @Nick: Indeed I mean Bias. $\endgroup$– DchucksCommented May 29, 2013 at 6:26
-
$\begingroup$ @gung: You are right, I am worried that some measurements are erroneous, especially that show sudden huge variation from a previous reading. $\endgroup$– DchucksCommented May 29, 2013 at 6:28
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
I get the impression you're talking about excluding outliers.
Exactly what should count as one (and whether they should be removed at all) sort of depends on your own views (like what 'accurate test result range' means) and your purposes.
It sounds like you might actually just need a reasonably robust estimate of scale; there are many such.
For example,
i) the quartiles for your data (using the default definitions in R) are 0.1090 and 0.2815; the interquartile range for your data is then 0.1725. The middle 50% of the data lie in between 0.1090 and 0.2815.
ii) the median absolute deviation from the median (MAD) is 0.0005; 50% of the data lie between median-MAD and median+MAD.
-
1$\begingroup$ Not to mention robust estimates of location. Trimmed means, for example, usually include more values than the median but fewer than the mean. (They equal the mean and median as limiting cases.) The 25% trimmed mean, or midmean, is easy to explain to users of boxplots as the mean of values inside the box, at least approximately. $\endgroup$– Nick CoxCommented May 28, 2013 at 14:16
-
$\begingroup$ +1 Yes, there's a lot to be said for trimmed means as simple yet robust location estimators; nice point about the midmean. $\endgroup$– Glen_bCommented May 28, 2013 at 15:10