7
$\begingroup$

I am involved in a project where I need to check whether my data follows a T-distribution with N degrees of freedom for a given value of N.

I know that Kolmogorov-Smirnoff can be used, but is there any test specifically tailored to testing for T distributions in particular. If there is nothing for T specifically, any test which works well for symmetric/unimodal distributions would help.

Thanks in advance.

$\endgroup$
  • 1
    $\begingroup$ We run KS-test and look at QQ-plots. $\endgroup$ – Aksakal Dec 4 '14 at 0:45
  • 5
    $\begingroup$ @Aksakal If you don't specify all the parameters before you see the data (or, equivalently if you center and scale the data by estimated parameter values), your p-values will be wrong; you'll be much less likely to reject than you should be. $\endgroup$ – Glen_b -Reinstate Monica Dec 4 '14 at 1:16
  • 5
    $\begingroup$ I wonder if this is really the question you needed to ask. I'm wondering if, instead, what you really want to know is whether some test statistic you're using really has a $t$ distribution. $\endgroup$ – rvl Dec 4 '14 at 3:03
5
$\begingroup$

Here's how to run KS-test on $t$-distribution.

  1. Suppose you have a sample which you suspect is from $t$-distribution, and has size = $n$
  2. Estimate the t-distribution parameters from the sample.
  3. Generate $M$ samples of size $n$ from the estimated distribution.
  4. For each sample obtain KS statistics using the estimated distribution as theoretical
  5. Build empirical nonparametric distribution from obtained statistics, e.g. using kernel density estimators
  6. Obtain KS statistics for the original sample and the estimated distribution
  7. Obtain $p$-value for the KS-stat using the empirical distribution of statistics
  8. Make decision based on confidence level

In your case d.f. is a given, so you can fit $t$-distribution with a given $N$ instead of estimating it like in my example here in MATLAB ('nu' variable is d.f.).

% True T-distribution
true_pd = makedist('tlocationscale','mu',0,'sigma',1,'nu',2);

% plot true distribution
x=0:0.01:1;
plot(icdf(true_pd,x),x);
hold on;
plot(norminv(x),x);
legend({'t' 'normal'},'Location','Best')
title 'CDF'

enter image description here

rng(0)
% obtain a sample
n=100;
sample = random(true_pd,n,1);

subplot(2,1,1)
histfit(sample,20,'normal');
title 'Sample from t(2,1,0) fit with Nromal'
subplot(2,1,2)
qqplot(sample);

enter image description here

% estimate H_0: T-distribution from this sample
disp 'H_0:'
null_pd = fitdist(sample,'tlocationscale')
[~,~,ksstat] = kstest(sample,'CDF',null_pd);

% get KS-test critical values by parametric bootstrapping from estimated
m=999;
r=random(null_pd,n,m);

stats = zeros(m,1); % store test statistics
est_pd = makedist('tlocationscale');
opts = statset(statset('tlsfit'),'MaxIter',1000);
opts = statset(opts,'MaxFun',2000);

for i=1:m
    bsample = r(:,i);
    [~,~,stats(i)] = kstest(bsample,'CDF',est_pd.fit(bsample,'options',opts));
end

p = (sum(stats>ksstat)+1) / (m+1);
mcErr = sqrt(p*(1-p)/m);
fprintf('KS stat: %f, p-value: %f, Monte Carlo error: %f\n',ksstat, p , mcErr);

% get the empirical distribution of KS test statistics
epd = ProbDistUnivKernel(stats);

% popular critical values
disp 'Crit. values for \alpha= 0.1, 0.05 and 0.01'
icdf(epd,[ 0.9 0.95 0.99])

figure
plot(icdf(epd,x),x)
grid on
title 'KS-test statistics simulated distribution'

enter image description here

OUTPUT:

H_0:

null_pd = 

  tLocationScaleDistribution

  t Location-Scale distribution
       mu = 0.161093   [-0.117585, 0.439771]
    sigma =  1.06958   [0.799248, 1.43136]
       nu =  1.58744   [1.02505, 2.45837]

KS stat: 0.041646, p-value: 0.865000, Monte Carlo error: 0.010812
Crit. values for \alpha= 0.1, 0.05 and 0.01

ans =

    0.0720    0.0791    0.0931

In this case based on $p$-value we can not reject that the sample comes from the t-distribution.

You can compare the critical values with the standard critical values here. You can replace the lines of code where I define and estimate $t$-distribution by the standard normal distribution, and see that the critical values match the table in my link.

If you don't like my method, you can follow this paper, which describes bootstrapping in detail: Jogesh Babu, G., and C. R. Rao. "Goodness-of-fit tests when parameters are estimated." Sankhya: The Indian Journal of Statistics 66 (2004): 63-74.

If you want to see what @Glen_b is talking about when saying that the distribution must be know and not estimated see item #3 here in the NIST Handbook.

It's not a very powerful test if your sample is normal and you have to estimate d.f. It's very difficult to distinguish between normal and $t$-distribution in small sample sizes, because $t$-distribution converges to normal when d.f. N$\to\infty$. In your case N is given, so the test should work fine.

$\endgroup$
  • $\begingroup$ Actually i have encountered the problem of KS test with unknown parameters as well. Aksakal, thanks a lot for the code ! However i have a dubt, the line: est_pd = makedist('tlocationscale'); maybe must be replaced with this est_pd = makedist('tlocationscale','mu',null_pd.mu,'sigma',null_pd.sigma, 'nu',null_pd.nu); $\endgroup$ – markowitz Feb 23 '16 at 17:48
  • $\begingroup$ @markowitz, I wrote the code long ago, you might be right. However, tLocationScale is not a "real" distribution :) It's simply stretched and shifted Student t. So, you can stretch and shift by sample variance and mean to get Student t, and that's probably what I'm doing $\endgroup$ – Aksakal Feb 23 '16 at 19:46
3
$\begingroup$

Unless you have prespecified the mean and variance before you see the data, a straight Kolmogorov-Smirnov test is not suitable - indeed once you estimate parameters it's no longer distribution free.

Your p-values will be quite wrong - your actual significance level will be much lower than your nominal rate and power will be correspondingly low.

If you want to do a Kolmogorov-Smirnov-like test, you need the t-distribution version of a Lilliefors test (essentially, a K-S test with fitted parameters).

One might do better with an adapted version of a Shapiro-Francia type test, based off the squared correlation between the observed values and approximate expected order statistics for a $t_N$ distribution. This will correspond directly to the correlation in a suitable Q-Q plot.

Distributions under the null can be simulated for either of the above tests.

Another alternative if sample sizes are not small is the Anderson-Darling test. It has the same issue as the K-S but the impact seems to drop off relatively rapidly with sample size. See the discussion in D'Agostino and Stephens "Goodness of fit Techniques"

[If you have a specific alternative in mind, (one in which the $t_N$ might be a special case, such as testing against a t with a different df) you may be able to make a likelihood ratio statistic.]

There are tests for unimodality and symmetry, as well.

$\endgroup$
  • $\begingroup$ Thanks. I will go with Aksakal's method for now. But will check out Anderson Darling test $\endgroup$ – Cosmonut Dec 4 '14 at 5:16
  • 1
    $\begingroup$ The Lilliefors test (KS with simulated distribution under the null) - which is what Aksakal is doing - should work just fine. $\endgroup$ – Glen_b -Reinstate Monica Dec 4 '14 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.