# Legitimacy of Excluding Extreme Values - A Specific Case

I'm handling surgical times: operating room time, time to do certain procedure, among others.

As you might guess, sometimes (hopefully rarely) surgeries have complications and thus aggravate immensely some of these surgical times, which then translates into a very rare but very extreme value (like - median 40 minutes, but these cases reach 200).

1) Is it expected for a variable such as surgical times, performed by the same team of surgeons on the same center during a not so long period of time, to have a normal distribution?

2) If my data is non-normally distributed, what statistical "excuse" / reason do I have to exclude these aberrant values?

2.1) For example, if the data is normally distributed, i would use the interquartile outlier labeling rule (Hoaglin) with k / the multiplier 2.2 as said "excuse"

2.2) But on the other hand, if I have non normal data, how could I justify, or better, is it statistically legitimate to exclude these values?

EDIT1: Adding clarity to the purpose of the study; i'm addressing the influence of a new surgical technique (for the same procedure)

EDIT2: as suggested, I am adding some data to better contextualize the question. I have included the two variables mentioned in the comments - Time1 (one of these surgical times in minutes) and Blood Units Consumption - do note that "999" corresponds to missing value! N=120 - 60 in each group.

1) As you might notice, specially in the "old technique" group, there are a few aberrant values - if I understood correctly, there are no "statistically" valid reasons to exclude them, correct?

2) Secondly, I often read that choosing a mean comparison statistical test based solely on Normality Tests (Shapiro-Wilk, for example), despite being often suggested by some textbooks and websites, might not always be the best approach, and that nothing replaces "the good sense of a statistician" - as I am not a statistician, would you be so kind to elaborate on the subject and, more specifically, perhaps exemplify using the data provided?

3) Nick Cox said "t test usually works well even with moderate non-normality." which i found to be a very interesting statement - care to explain, please?

This has been really helpful, thank you all in advance!

Old0New1Technique  Time1   BloodUnits

0      36         3

0      52        34

0      52        30

0      36         2

0      38         6

0     110        16

0      45         8

0      40         0

0      40       999

0      42        16

0      81       129

0      74        19

0      44        26

0      28         4

0      44        18

0      46        19

0      43        18

0      36         7

0      40        29

0      36        14

0      65        34

0      68        21

0      35        15

0      60        56

0      43         9

0      39        10

0      39       999

0      18         1

0      44        14

0      42        53

0      42        53

0      53        48

0      36        16

0      70        28

0      34        28

0      41         2

0      30         0

0      44         0

0      31         2

0      38         2

0      43         5

0      35        31

0      38        28

0      30         2

0      37        21

0      45         4

0      38       999

0      43         1

0      41         2

0      55        34

0      51         9

0      62         4

0      47        16

0     124       166

0      55        14

0      38        16

0      50        31

0      42        15

0      36        16

0      39        11

1      47        12

1      40         0

1      75         8

1      52         0

1      50         0

1      55         3

1      43        17

1      53         1

1      56         1

1      39         0

1      53         9

1      54         2

1      47         7

1      48         0

1      51        11

1      50         4

1      81         1

1      56         2

1      54         0

1      43         0

1      33         6

1      49         2

1      42         7

1      62         0

1      50         0

1      58         4

1      68         0

1      46         3

1      45         0

1      42         0

1      73         3

1      45         0

1      54        17

1      48         7

1     189         1

1      47         9

1      47         5

1      35         0

1      45         0

1      50         1

1      47         0

1      45         2

1      47         3

1      85         7

1      49         1

1      41         1

1      90         0

1      40        12

1      45         4

1      37         4

1      50         0

1      55         0

1      50         3

1      58         0

1      47        10

1      45         5

1      55         0

1      39         0

1      43         0

1      60         0


https://www.dropbox.com/s/4lp97nsv6f2jg99/SurgicalTechniqueDataSet.xlsx?dl=0

https://drive.google.com/file/d/0ByeSGirYFFwiUTkyMDNxV1ZWWjA/view?usp=sharing

• Strictly speaking, time, as in duration, could never be normally distributed because it's bound at zero (i.e., you could never have a procedure that took -10 minutes). You'd probably expect the distribution of procedure times to have a long tail Jan 24, 2017 at 12:18
• I wouldn't expect surgical times to be normally distributed in most cases. If there is a standard, predictably simple and rarely problematic procedure that might be approximately true. (Example: reversing dislocation of a shoulder.) For most procedures, I would expect that times would be definitely non-normal. There is no reason or excuse to throw out data that seem unusual unless you know independently that something quite special happened (e.g. a procedure was interrupted for some completely extraneous reason). Jan 24, 2017 at 12:19
• Excluded from what? I don't believe this question is answerable - except in a vague, general fashion - unless you edit it to explain what subsequent analysis you're proposing to carry out, to what purpose. Jan 24, 2017 at 12:21
• I would add to Nick Cox and Scortchi's comments that without knowing what the research question (RQ) or intended analysis is, these "outliers" (note the quotation marks) might be the datapoints of interest. Think about the RQ: "what triggers extremely lengthy operations?" excluding these datapoints when trying to answer this question is non-sensical as these might be the outcomes you were looking for in the first place.
– IWS
Jan 24, 2017 at 12:28
• A t test usually works well even with moderate non-normality. You could try a generalised linear model with technique as a binary predictor and test whether different link and family assumptions are crucial. I think you'd be likely to get an answer if you posted your data or otherwise made this really concrete. Otherwise we're speculating from other experience on what might or might not be good enough for your situation. Jan 24, 2017 at 15:07

## 2 Answers

If you are evaluating a new surgical technique then you should include the possibility of "complications" in your model. The new technique may alter the probability of complications and the time to deal with complications, not just the time to perform the procedure when there are no complications. You need to include those aspects in your model. This might not be so simple as a model based on log-normal or other simple distributions of procedure times, but it seems to be required for a fair comparison of the new technique against the present standard of care.

• You make a valid point. Perhaps it would be wrong of me to simply discard and never mention these values as they probably coincide with complicated cases that should be reported and compared between groups. But then again, I could exclude them if I wanted to make an analysis on the procedures without complications, which are more frequent and more reproducible. Jan 25, 2017 at 13:15
• @Karpad: That's fair enough as far as it goes, but it needs to make sense to view a surgical procedure as either involving a complication or not (rather than viewing all as involving varying degrees of complication), and either those involving complications need to be recorded as such, or it needs to be so clear from their duration which they are that you can pick them out with a negligible chance of making a mistake (if you have to resort to mucking about with outlier tests, it's not clear - how will you take that uncertainty into account in your analysis?). Jan 25, 2017 at 14:15

Typically "waiting times" (or "processing times") are modelled using an exponential distribution. An exponential random variable is positive, its distribution has a nice closed form, models the long tails that you are finding in your data, and is very well studied.

• I'd want a really good argument that the modal duration of an operation was zero time, even as an approximation. Jan 24, 2017 at 17:30
• Sure, if you are trying to model it exactly. For the purpose of outliers, the exponential is a simple start. A better fit would be a Beta or Gamma distribution. Jan 24, 2017 at 18:13