Suppose we are given a matrix of many rows (different genes for example) and few columns (different time points) and we want to identify the top rows (genes) that are following a trend, like monotonically increasing or decreasing. What is a good method to perform this in R?

The answers providing also a statistical significance result (something like p-value) are more than welcome. Finally the methods to indicate more complicated cases - such as up-then-down treands - are really perfect.

  • $\begingroup$ How many columns? Is there any missing data? $\endgroup$
    – Peter Flom
    Oct 27 '12 at 20:13
  • $\begingroup$ @PeterFlom Thanks for your attention. The number of columns is as less as 4 to as much as 10 in some cases. There are no missing datas fortunately $\endgroup$ Oct 27 '12 at 20:45
  • $\begingroup$ I don't think there's an easy answer, as you want to find monotinical phenomena, and then "bumpy". What I would do, is start with a regular linear model, looking for a straight line. Betas for parameters would indicate the trend. Then I would make the above linear function into a polynom, looking for curvy trends. Every step would be followed by a rigorous model checking. $\endgroup$ Oct 30 '12 at 8:13

You didn't mention the type of data you are identifying (continuous, discrete?), but, a simple method to approach this would be to re-code the data into binary values and sum each row to get the cumulative sum of the magnitude of steps -- the result is your trend strength. Once you have the run data, you can easily rank sort it and bin into percentiles of top and bottom trends per your cutoff thresholds (can use scale and qnorm and set to desired p-values accordingly if desired).

 rnd2bin<-function(x) ifelse(x>=0,1,-1)


 # find bottom and top 10% trends
 bottomtrend<-which(run < cut[1])
 toptrend<-which(run > cut[2])

> # display top and bottom 10% trend indices


10% 90%

-4 4


[1] 2 13 19 21 24 28 43 84


[1] 12 22 38 62 76

*The above code assumes all columns are populated, but possibly (depending on your data) an approach would be to just set any empty columns to zero assuming time intervals are all aligned (this obviously, may not be the best, but it depends on the nature of the data). Without seeing the data, I can't point out a best method, but if they had no time alignment, then one could argue the shorter series with all ones is just as trending as the longer series with all ones. Some type of calibration needs to be applied in that case.

** as mentioned above, you can set threshold to p-values by scale and qnorm. e.g.

pbottomtrend<-which(z < pcut[1])
ptoptrend<-which(z > pcut[2])
  • $\begingroup$ Thanks. I mentioned the data is gene expression that is normally continuous quantitive data. I have already performed a similar method - as you mentioned - to sort the genes based on an scoring function, but the problem is the statistical parameters like p-value, as I asked in the original question. $\endgroup$ Oct 30 '12 at 7:50
  • $\begingroup$ I modified to show the example as I suggested (for p-values). Could you clarify how it is different than what you are looking for? $\endgroup$
    – pat
    Oct 30 '12 at 8:48
  • $\begingroup$ I am still afraid the scoring model is based on zero-one data, while the original data are continuous $\endgroup$ Oct 30 '12 at 8:53
  • $\begingroup$ Something like Mann-Kendall Trend Test (in kendall package) might be more suitable- with ranking by p-values? $\endgroup$
    – pat
    Oct 30 '12 at 15:58
  • $\begingroup$ Do you believe this test is suitable for this amount of time poitns? $\endgroup$ Oct 30 '12 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.