Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

This question is related to Excel's CHIDIST function in MATLAB

The minimum answer larger than zero I can get in Excel is about 2.2e-308 that correspond to realmin in MATLAB. However, in MATLAB the minimum answer is 2.2e-16, or eps(1.0). I understand this is because I first calculate chi2cdf that is close to 1 for large values, and the precision in this case is eps(1.0).

Is there any other way to reproduce Excel's CHIDIST and not to loose the precision? Do you know exactly how CHIDIST works?

EDIT:

I have to do this test multiple times, and in many cases the resulting p-values are close to zero. I need to rank the p-values, this is why I need high precision in p-value calculation.

share|improve this question
I am assuming that this is merely out of curiousity, as I can't imagine a "real" example where something like this matters. So if you are worried about this loss of precision for some real world problem, don't be. It is still interesting to know these sorts of things though :) – probabilityislogic Aug 6 '11 at 18:25
I'm unfamiliar with Matlab, but I'm confident it can calculate an incomplete Gamma integral to very high precision. See the 'upper' option at mathworks.com/help/techdoc/ref/gammainc.html. – whuber♦ Aug 6 '11 at 18:38
@probabilityislogic: Actually I do need this precision. Later I take log of p-values, so I get many Inf values, if p-value is 0. – yuk Aug 7 '11 at 2:23
@whuber: You are right. CHI2CDF uses GAMMAINC, which in its turn takes integral from 0 to x. As I found (wiki.services.openoffice.org/wiki/Documentation/How_Tos/…) CHIDIST takes the same integral but from x to Inf. The upper option is exactly what I need. Please it as an answer. – yuk Aug 7 '11 at 2:39
@yuk - You should really add this information to the question, that what you really require is a good approximation to the log of p-value. In any event, I still struggle to see why not simply giving an arbitrarily big value whill not suffice. Your minimum value is $-16\log(2.2)$, so just take $\log(0)=-20$ or bigger – probabilityislogic Aug 7 '11 at 3:45

1 Answer

up vote 1 down vote accepted

A solution was developed in the comments following the question; the purpose of this post is to make that solution available as a reply so it can be searched, voted on, etc.

Chi-square distribution functions are scaled incomplete Gamma integrals. If you compute the integral from $0$ to $x$, you approach $1$ as $x$ gets large. Subtracting this value from $1$ loses ever more precision; with double-precision arithmetic, you're limited to results greater than about $10^{-16}$. The trick is to compute the integral from $x$ to $\infty$, which typically can be approximated accurately even for extremely small values.

The Matlab help for the incomplete Gamma integral indicates that this upper tail will be computed using its 'upper' option.

share|improve this answer
1  
Thank a lot! To be more MATLAB specific, function chi2cdf calles gamcdf, which calles gammainc. In gamcdf I've changed the line p = gammainc(z, a); to p = gammainc(z, a, 'upper');, saved it under another name and changed the call in chi2cdf. Works great! – yuk Aug 8 '11 at 3:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.