I have tried to search, but couldn't find an answer for my problem.

I have data from different subjects that are exposed to different conditions. But they are exposed to the different conditions unequal times (sometimes even zero times). Like this

           Condition 1               Condition 2                      Condition 3   


Subj 1 |   Meas1 Meas2 Meas3         Meas1 Meas2 Meas3 Meas4 Meas5    Meas1 Meas2 Meas3

Subj 2 |   Meas1                     Meas1 Meas2 Meas3                Meas1 

Subj 3 |   Meas1 Meas2 Meas3 Meas4   Meas1                            Meas1 Meas2 Meas3

Subj 4 |    ---                      Meas1 Meas2 Meas3 Meas4          Meas1 

Subj 5 |   Meas1 Meas2 Meas3         Meas1 Meas2 Meas3                Meas1 Meas2 


I want to test (non-parametrically), if the measurements under the conditions are different, like in an Kruskal-Wallis or Friedman test, but I want to keep the information from the pairing by subjects. And I don't want to have to discard measurements just because there were no measurements for one of the conditions.

What kind of test am I looking for?

Also, if it can be done in Matlab, I would be really happy!

  • $\begingroup$ Newb here. What do you mean by non-parametric. They all have parameters. Some have many more parameters than samples. They can be called hyper-parametric, or more informatively model-agnostic. ... My first thought is to make some variation on a cube plot. I like to visualize the data. EDA, right. Good luck. $\endgroup$ Aug 20, 2013 at 14:03

1 Answer 1


I doubt you will find a specific non-parametric test developed for this purpose, as such problems are generally tested using variants of regression. Which type of model you will need is going to depend a bit upon what you mean by "non-parametrically".

If the reason that you want a non-parametric test is because your measurements are ordered categorical measurements (e.g., a rating), then you need to fit a limited dependent variable model (e.g., an ordered probit or logit model), and you would use whatever test is built into the software you use to test for the difference in means (e.g., a likelihood ratio test). I do not use MATLAB, but this model is available in every program that I do use (R, Q, LIMDEP, SPSS).

If the reason that you want a non-parametric test is because you are concerned about other violations of assumptions then you should considered using one of the robust regressions (e.g., M-Regression).

Whatever method you end up using, you will end up discovering that you are going to have to make some quite strong assumptions about the nature of the missing data (e.g., is it Missing At Random?) and about correlations between the measurements within each subject. That is, the nature of your data means you are not going to find any simple "test" that you can apply without getting quite deep into the technicalities of the modeling process. This is because your data is much more complicated than the types of data that apply to the Kruskal-Wallis or Friedman test, and this complexity in data leads to complexity in data analysis.

However, if time is of the essence and you don't have the resources to get deep into the modeling side of things, you should get some insight by:

  • Computing a mean measurement for each respondent and then testing these with Kruskal-Wallis
  • Counting up the number of measurements for each respondent and testing the difference in the number of counts using Kruskal-Wallis.

If the second of these tests is not significant it gives you some confidence that the first of the tests is informative. Having said that, the "right" way is to build a proper model.

  • $\begingroup$ Thanks so much! What I mean by non-parametric is, that the means of measurements pr. subject are not normally distributed. The suggestions are good, but somehow I feel like I'm losing information, if I just average the measurements for each subject. The subjects where we have many measurements should carry more "weight" as they do contain more information. And the data is randomly missing. Hm. $\endgroup$
    – Chrelli
    Aug 20, 2013 at 14:54
  • $\begingroup$ You are definitely losing information with the simple methods I propose. Building a hierarchical regression model is the way to go if this is concern. If you want to give the average Measurements from subjects with more response a greater weight then you can apply a weight in the analysis. This Works in SPSS my own product, Q, but you would want to run the analysis weighted and unweighted in whatever program you use and logic check the results because different programs treat weights differently. $\endgroup$
    – Tim
    Aug 28, 2013 at 4:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.