Return to Answer

1. A practical suggestion.

Change this part of the code

    if mad == 0:
        mad = 9223372036854775807 # maxint

to

    if mad == 0:
        mad = 2.2250738585072014e-308 # sys.float_info.min

It does the trick. Division by this number blows up the Iglewicz-Hoaglin test statistic – exactly as desired. That is, marking strongly deviant observations as outliers.

2. Previous practical suggestion.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

IBM uses this variant:

$$ M_{i} = \frac{x_{i} - \text{median}(x)} { 1.253314 \cdot \text{MAE} } $$

for the if MAD == 0 case.

3. What is going on here? (From a programming perspective)

Consider the the two cases:

1. $0/0$,
2. $x/0$ for $x \neq 0$.

Scientific programming languages R, Matlab and Julia have the following behavior:

1. 0/0 returns NaN.
2. 10/0 returns Inf.

Python, on the other hand, throws a ZeroDivisionError in both cases.

Practical suggestion one circumvents both cases for both flavors of zero-division handling.

1. A practical suggestion.

Change this part of the code

    if mad == 0:
        mad = 9223372036854775807 # maxint

to

    if mad == 0:
        mad = 2.2250738585072014e-308 # sys.float_info.min

It does the trick. Division by this number blows up the Iglewicz-Hoaglin test statistic – exactly as desired. That is, marking strongly deviant observations as outliers.

2. Previous practical suggestion.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

IBM uses this variant:

$$ M_{i} = \frac{x_{i} - \text{median}(x)} { 1.253314 \cdot \text{MAE} } $$

for the if MAD == 0 case.

3. What is going on here? (From a programming perspective)

Consider the two cases:

1. $0/0$,
2. $x/0$ for $x \neq 0$.

Scientific programming languages R, Matlab and Julia have the following behavior:

1. 0/0 returns NaN.
2. 90/0 returns Inf.

Python, on the other hand, throws a ZeroDivisionError in both cases.

Practical suggestion one circumvents both cases for both flavors of zero-division handling.

A practical suggestion.

Change this part of the code

    if mad == 0:
        mad = 9223372036854775807 # maxint

to

    if mad == 0:
        mad = 2.2250738585072014e-308 # sys.float_info.min

It does the trick. Division by this number blows up the Iglewicz-Hoaglin test statistic – exactly as desired. That is, marking strongly deviant observations as outliers.

Previous practical suggestion.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

Or reserve this tweak just for the if MAD == 0 case.

1. A practical suggestion.

Change this part of the code

    if mad == 0:
        mad = 9223372036854775807 # maxint

to

    if mad == 0:
        mad = 2.2250738585072014e-308 # sys.float_info.min

It does the trick. Division by this number blows up the Iglewicz-Hoaglin test statistic – exactly as desired. That is, marking strongly deviant observations as outliers.

2. Previous practical suggestion.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

IBM uses this variant:

$$ M_{i} = \frac{x_{i} - \text{median}(x)} { 1.253314 \cdot \text{MAE} } $$

for the if MAD == 0 case.

3. What is going on here? (From a programming perspective)

Consider the the two cases:

1. $0/0$,
2. $x/0$ for $x \neq 0$.

Scientific programming languages R, Matlab and Julia have the following behavior:

1. 0/0 returns NaN.
2. 10/0 returns Inf.

Python, on the other hand, throws a ZeroDivisionError in both cases.

Practical suggestion one circumvents both cases for both flavors of zero-division handling.

Just a practical suggestion here.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

Or reserve this tweak just for the if MAD == 0 case.

A practical suggestion.

Change this part of the code

    if mad == 0:
        mad = 9223372036854775807 # maxint

to

    if mad == 0:
        mad = 2.2250738585072014e-308 # sys.float_info.min

It does the trick. Division by this number blows up the Iglewicz-Hoaglin test statistic – exactly as desired. That is, marking strongly deviant observations as outliers.

Previous practical suggestion.

What you could do, is check if it works with the closely related definition of mean absolute error (MAE):

$$ \text{MAE} = \frac{1}{n} \sum_{i=1}^n |x_i - \text{median}(x)|, $$

with $e_i = x_i - \text{median}(x)$ the errors (better: residuals, or, deviations).

Or reserve this tweak just for the if MAD == 0 case.