0
$\begingroup$

I have some matlab code given from my advisor at university to check if my data is normal distributed:

test_cdf = makedist('tlocationscale','mu',mean(data),'sigma',std(data),'nu',1);
[h,p] = kstest(data,'CDF',test_cdf) 
% h = 0, p = 0.2131

Documentation of kstest (One-sample Kolmogorov-Smirnov test)

From this he is concluding that the data is NOT normal distributed. I do not understand his reasoning, but he expects me to understand it on my own and I have to write about and explain it. Hours of googling did not help me. :-/

What I understand is that p > 0.05, consequently the null hypothesis cannot be rejected, that is why h = 0. Null hypothesis in this case means that data comes from the Student's t distribution. But why does it mean that data is NOT normal distributed? Is his conclusion correct?

I am especially wondering, since the following seems to imply normal distribution for my data, doesn't it?

test_cdf = makedist('Normal','mu',mean(congru),'sigma',std(congru));
[h,p] = kstest(data,'CDF',test_cdf) 
% h = 0, p = 0.2952

Thank you very much for your help in advance!

Update:

What I can see now is that normal distributed random numbers behave differently with respect to the first code example:

rng(0,'twister'); % normal dist random numbers
rnd_data = 4+randn(1,10000);
test_cdf = makedist('tlocationscale','mu',mean(rnd_data),'sigma',std(rnd_data),'nu',1);
[h,p] = kstest(surdata,'CDF',test_cdf)
% h = 1, p = 9.0599e-143

Can it be that rejection of the null hypothesis "data comes from the Student's t distribution" means that the data follows a normal distribution? If yes why?

$\endgroup$
1
$\begingroup$

I can't give you a mathemathical answer and I don't use matlab, but I think your advisor is wrong in this case.

If I understand you correctly, from the comment below, your advisor tested your data against a student's t-distribution and the p-value was about 0.21, which means that you cannot reject the null hypothesis that your data does follow a t-distribution. Your advisor means that this indicates that this means that your data follows a t-distribution.

But your advisor is wrong, the test only says that the null hypothesis cannot be rejected, not that it should be accepted. In fact, this is easy to see if you did your test first: you test against a normal distribution and cannot reject the null hypothesis, is your data therefore normally distributed and your supervisor is wrong? No, of course not. Both tests just say that there is insufficient evidence to reject either of these null hypothesis, but they say little about the distribution of your data itself (other than it must resemble both these distributions to get these results).

You could simulate a variable that follows a normal distribution with 30 observations or so, and when you test it against a t-distribution the p-value will often be much higher than 0.05. This simply means that you cannot reject the null hypothesis that the data follows a t-distribution, but you cannot conclude that it does follow the t-distribution.

This problem is in fact all over science when we have negative results. Absence of a significant difference between groups etc. does not mean that there is no difference, only that the null hypothesis that there is no difference cannot be rejected (typically at the .05 level).

$\endgroup$
  • $\begingroup$ Ok, so you are referring to the 2nd example code from above. I have exactly the same thoughts on this as you. My problem is the 1st code example. I do not understand it thoroughly. Maybe it is an argument for not normal distributed? $\endgroup$ – Gerhard Hagerer Sep 4 '15 at 10:43
  • $\begingroup$ Actually, I wasn't referring to any code as I don't use matlab. I was just talking about the KS test. $\endgroup$ – JonB Sep 4 '15 at 10:45
  • $\begingroup$ Ok, to hold it more generally: Can it be that acceptance of the null hypothesis that the data follows Student's t distribution is an argument for NOT normal distributed (if tested with KS of course)? $\endgroup$ – Gerhard Hagerer Sep 4 '15 at 10:52
  • $\begingroup$ Ah, I see! I'll update my answer. $\endgroup$ – JonB Sep 4 '15 at 11:11
  • $\begingroup$ Is it possible there is a typo in paragraph two of your answer? "[...] the p-value was about 0.21, rejecting the null hypothesis that your data does follow a t-distribution." Should mean "not rejecting", doesn't it? $\endgroup$ – Gerhard Hagerer Sep 4 '15 at 11:36
1
$\begingroup$

Your advisor is most likely right, because he's your advisor. This statement may sound odd, but it is more likely that you:

  • did not convey to us correctly what your advisor saying
  • missed a context of your advisor's statement
  • failed to convey some other things your advisor said in connection to this example etc.

The first snippet of code which you produced has two issues.

The main issue is that you estimated the parameters of the distribution, then you use them to construct null hypothesis. That can't be done with KS test, see item #3 here. In KS test your null hypo should not come from data itself. If you want to use KS like test, then take a look at Anderson-Darling or Lilliefors tests in MATLAB, they overcome the issue I noted. (A small issue with these functions is that it doesn't support Student t out-of-the-box).

The second problem is that formally when you conduct the hypo test you need two hypos: $H_0$ and $H_a$. So, the complete approach would have been to claim that $H_0$ = data comes from Student-t, $H_a$= data comes from normal. Then you run the test against $H_0$, get p=0.21, and fail to reject the null (Student-t). This doesn't formally mean accepting Student t, and the theorists will go nuts about this facts. However, in practice you have to decide whether you use Normal or Student t. So, after running a test and getting fail to reject, you actually go with Student t. Hence, in this regard for all practical purposes you do accept Student t. This is not pure stats though, this is decision making.

Having said all of this, my guess is that your advisor is probably not a rookie, and you might have misunderstood him. So, go and talk to him.

$\endgroup$
  • $\begingroup$ Questioning my advisor's approach is legetimate, see Berge's answer. My advisor sent me source code from the kstest-documentation without telling or explaining where it was taken from nor what it does. He repeatingly rejects talking about it. So I doubt that your assumptions about me are correct. The code snippets were not produced by my advisor as mentioned initially. So far about the overhead. Apparently rejection of the null hypothesis "data is from Student's t distribution" for my advisor means the data follows a normal distribution. Do you see this as reasonable way of decision making? $\endgroup$ – Gerhard Hagerer Sep 4 '15 at 15:02
  • $\begingroup$ @GerhardHagerer, this is how it goes: $H_0$=normal, $H_a$=Student t. You run the test, get very low p-value, so you "reject $H_0$, and accept H_a". You seem to keep forgetting about the role of alternative hypo in inference. $\endgroup$ – Aksakal Sep 4 '15 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.