3
$\begingroup$

I'm currently reading the paper Distribution of Variables by Method of Outlier Detection and am trying to understand the section on Minimum generalized variance

If my understanding is roughly correct, it basically starts off with no points and then adds points to a data-set. Each point is then assign a variance score. The further away the data point is from the median the more variance it accrues. The paper does a much better job at explaining it and is very accessible. My question is concerned with the section on cutoffs. For multi-variate outliers it uses the formula

$M_{G}+\sqrt{\chi_{0.975 . p}^{2}}\left(q_{3}-q_{1}\right)$

I understand the role of q1 and q2 but could anyone explain to me what the $\chi_{0.975 . p}^{2}$ in the section of the formula represents?

$\endgroup$
2
$\begingroup$

Comment: Short answer: I don't really know. Explorations follow.

In the linked paper, it seems early on that $p$ is degrees of freedom (or observations or dimensions). Then just after display (6) the author seems about to define $\chi_{0.975,p}^2$ when the sentence ends abruptly. At (7) this expression occurs again.

My guess is that this is the value that cuts probability 0.025 from the upper tail of $\mathsf{Chisq}(\nu = p).$ Taking $p=2$ in (7) this would lead to $\sqrt{\chi_{0.975,p}^2} = 2.716,$ which might make sense in context. Computation in R:

sqrt(qchisq(.975, 2))
[1] 2.716203

However, a more common use of subscript notation with $\chi^2$ is that the probability in the subscript is cut from the lower tail of the distribution (a 'percentage point'). Then the quantity would be $0.225.$

sqrt(qchisq(.025, 2))
[1] 0.2250236

Note: The author quotes several well-known statisticians as being opposed to the removal of outliers, unless traced to a specific mistake. Then seems to ignore this advice with various suggestions for detecting and removing inconvenient outliers from multivariate data.

$\endgroup$
  • 1
    $\begingroup$ Hi @BruceET, Thank you very much for taking the time out to explore the question. I went to the end of the research paper and found the mgv.output < -outmgv(full.data,y = NA,outfun = outbox) which appears to come from this book. It seems to be the square root of the .975 quantile where p is the degrees of freedom (pg 230) $\endgroup$ – John Smith Aug 12 '19 at 7:50
  • 1
    $\begingroup$ The formula in the question is a familiar one in nonparametric statistics: one computes or estimates a "pseudo-standard deviation" by means of a robust statistic such as the IQR (as in this case) and then rescales it by a universal constant so that the pseudo-SD for a Normal distribution equals the usual SD. I believe that's all that's going on here. $\endgroup$ – whuber Aug 12 '19 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.