I have a table like below (it is a small subset of my data. In this table, I measured one variable over 4 different time points (T1,..., T4), now I would like to check is there any significant difference between time points for each sample? then based on that I will select those samples that have variability in different time points.

My assumption for the data is:

  • non-normal distribution.
  • unequal variance.
  • the same sample size for each dependent group.

I have reviewed several methods (like, Repeated Measure ANOVA, GLM, GEE, linear mixed model, Kruskal Wallis test and GLMM), but I am confused about which one is more appropriate for my data?

 sample                                T1                       T2                        T3                       T4

1:824850-825300                 0.00000000                0.0000000                0.0000000                0.0000000        
1:894445-894831                 5.39848590                3.9919398                5.8171244                3.4732853         
1:902180-902369                 5.30856403                4.7035677                1.6972109                4.0094193
1:911400-911969                 3.93351892                8.6449756                3.9462391                5.9417675
1:912000-912125                 3.08713416                3.7929570                0.5132366                2.7979578
1:919425-920025                 4.37344006                6.4203699                3.5285015                3.4974473
1:934044-934294                 9.87882930               11.3788710                7.4419304                6.0622420
1:948960-949100                 1.65382187               11.0063484                5.4989633               12.4908832
  • $\begingroup$ What's the nature of your dependent variable? What are you measuring? $\endgroup$
    – Stefan
    Commented Feb 7, 2020 at 14:43
  • $\begingroup$ Thanks for the reply! Actually, my data is including different histone marks (H3k27 ac, H3k4me1 and ...)that I have normalized data for each histone mark from different time points (here I only showed one histone mark and its different time point). I would like to measure is there any difference between different time points for eachsample, then if there is I select that row (sample) as variable sample. $\endgroup$
    – star
    Commented Feb 7, 2020 at 15:01

1 Answer 1


Your data is setup in a "wide" format, which means that you have separate variables associated with each time point. This is fine if you want to use something like structural equation modeling or certain classical tests (ANOVA). But to use GEE or GLMM, you want to set your data up in a long format. For example, using just the first two samples, it would look something like this:

sample         time  outcome 
824850-825300   0    0.00000
824850-825300   1    0.00000
824850-825300   2    0.00000
824850-825300   3    0.00000
894445-894831   0    5.39848590
894445-894831   1    3.9919398
894445-894831   2    5.8171244
894445-894831   3    3.4732853

Once your data is setup like this, you can use GLMM or GEE to estimate a model that uses data from all samples. However, it seems like you want to do some sort of selection of samples to analyze beforehand, and that is confusing to me. First of all, all the methods you have identified operate on the entire collection of samples in your data. They don't identify whether any one sample within the collection of samples in your data are somehow different from one another.

An example of a very basic mixed model you could use on your data, and using a linear (Gaussian) mixed model in R, is the following:

m <- lmer(outcome ~ 1 + (1|sample), data=df)

This model allows for unique intercepts (mean value on the outcome) for each sample. The idea is that values within samples are likely to be correlated and you want to allow for such correlation. From this model you get a variance estimate for sample ($\sigma^2_j$) and the residual ($\sigma^2_i$), representing between- and within-sample variation, respectively. You can quantify the average correlation between outcome values within a sample by calculating an intraclass correlation coefficient (ICC): $\sigma^2_j$/($\sigma^2_j$ + $\sigma^2_i$). Again, this would be a correlation at the whole sample level.

Typically, you would want to use predictors to explain variation between and within samples, including predictors that are constant within a sample (time-invariant) and predictors that are unique to each time point within a sample (time-varying). Perhaps this is not the answer you are looking for. If that is the case, then provide greater detail on what you want to do.

  • $\begingroup$ Thanks for your reply. I made a table as you mentioned. But I would like to know is there any significant difference between the time point for each sample. e.g: for 824850-825300 sample as you see all values are zero and for 894445-894831 sample there is more variability. so I would like to have some confident statistics like p-value, FC or etc, as a scale to separate informative samples. $\endgroup$
    – star
    Commented Feb 11, 2020 at 13:50
  • $\begingroup$ Can you tell us the ICC fro the m model? I'm curious if almost all of the variation is between samples rather than within them. This would be the case if the ICC was large (say > .7). $\endgroup$
    – Erik Ruzek
    Commented Feb 11, 2020 at 14:21
  • $\begingroup$ it is 0.7136501. $\endgroup$
    – star
    Commented Feb 11, 2020 at 14:59
  • 1
    $\begingroup$ That is fairly large. The ICC can be interpreted as the correlation between two randomly chosen observations for a randomly chosen sample. In other words there is not a lot of within-sample variation; most variation in your data is between different samples. I'm not sure there is therefore much benefit in trying to identify samples that are more heterogeneous. Even at that, I am not certain how you would do that. You could compute a standard deviation or other measure of spread for each sample and then choose a cutoff value for inclusion into your subsample. $\endgroup$
    – Erik Ruzek
    Commented Feb 11, 2020 at 16:26
  • $\begingroup$ Thanks for your helpful explanation. $\endgroup$
    – star
    Commented Feb 11, 2020 at 16:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.