2
$\begingroup$

I am having difficulty finding out what analysis suits my question best. My data can be found at: https://docs.google.com/spreadsheets/d/1roKj3DBEI05d6aatqA5Ge0mSIzPeBVTTXgDjaqCJ2fQ/edit?usp=sharing

It looks like this:

# A tibble: 132 x 65
   category channel   PPN1   PPN2    PPN3   PPN4  PPN5   PPN6   PPN7   PPN8     PPN9  PPN10  PPN11  PPN12
      <dbl>   <dbl>  <dbl>  <dbl>   <dbl>  <dbl> <dbl>  <dbl>  <dbl>  <dbl>    <dbl>  <dbl>  <dbl>  <dbl>
 1        0       1 -20.4   49.2   -38.0   51.9  -25.5   2.85  21.1   113.  -151.    -180.  417.   -12.0 
 2        1       1 -21.7   16.7    -2.34  -7.73  22.9 -41.1   33.6  -195.  -101.     -72.5 146.   -70.4 
 3        2       1  24.8   24.1   -41.7  -29.2   40.1 -20.4    8.00 -288.   -94.8     44.3   8.41 -47.0 
 4        0       2  -1.40 -36.0    19.7   52.3   30.9  18.5  -29.1   544.   101.     273.  101.     2.69
 5        1       2  13.9  -48.0   -19.9  -34.5  -19.1  28.6  -32.0  -251.   167.      59.9  83.1   51.3 
 6        2       2 -44.9  -26.7    39.5   10.8  -38.1  10.5  -46.9   -79.0  119.     -15.0 116.     3.01
 7        0       5 -10.9    7.96   31.2  -83.9  -30.4  41.6  -35.0   616.  -199.     175.   11.7   22.1 
 8        1       5 -36.0    1.32   31.3  -69.5  -10.7 -65.9   37.4  -252.     0.687  -32.7 141.   -39.2 
 9        2       5 -33.9  -21.8     8.97 -45.5   12.5  -3.43  17.2  -206.   -60.8    -36.2 102.   -38.5 
10        0       6  86.1   -6.47 -114.   158.    61.0 100.   -74.2    14.4   -7.63   -15.1 -70.2  -39.8 
# … with 122 more rows, and 51 more variables: PPN13 <dbl>, PPN14 <dbl>, PPN15 <dbl>, PPN16 <dbl>,
#   PPN17 <dbl>, PPN18 <dbl>, PPN19 <dbl>, PPN20 <dbl>, PPN21 <dbl>, PPN22 <dbl>, PPN23 <dbl>, PPN24 <dbl>,
#   PPN25 <dbl>, PPN26 <dbl>, PPN27 <dbl>, PPN28 <dbl>, PPN29 <dbl>, PPN30 <dbl>, PPN31 <dbl>, PPN32 <dbl>,
#   PPN33 <dbl>, PPN34 <dbl>, PPN35 <dbl>, PPN36 <dbl>, PPN37 <dbl>, PPN38 <dbl>, PPN39 <dbl>, PPN40 <dbl>,
#   PPN41 <dbl>, PPN42 <dbl>, PPN43 <dbl>, PPN44 <dbl>, PPN45 <dbl>, PPN46 <dbl>, PPN47 <dbl>, PPN48 <dbl>,
#   PPN49 <dbl>, PPN50 <dbl>, PPN51 <dbl>, PPN52 <dbl>, PPN53 <dbl>, PPN54 <dbl>, PPN55 <dbl>, PPN56 <dbl>,
#   PPN57 <dbl>, PPN58 <dbl>, PPN59 <dbl>, PPN60 <dbl>, PPN61 <dbl>, PPN62 <dbl>, PPN63 <dbl

I want to predict category based on the values under all subjects (PPN1,PPN2..). I also want to know what Channels are best at predicting this.

What analysis (in R) would you recommend?

I have looked into Repeated Measures ANOVA, mixed effect modelling (category as fixed and subject ID as random effect) and Linear Discriminant Analysis but none have satisfied my needs (yet).

This is my first statistical analysis so it might be a simple question. Thanks for your time!

Improve this question
$\endgroup$
0
$\begingroup$

I'm not sure exactly what you mean by knowing what channels are best at predicting the category. I am guessing you mean determine which channels have the highest rates of correct predictions after you use the other columns for predicting category. This may be an issue due to the small sample size per channel.

However, on the topic of predicting the category based on all the other columns, have you considered a Random Forest or similar model? RF is a tree based model meaning each column basically acts as fork in a road. In the simplest case possible, a tree could come to PPN1 and see the value is greater than 0 and then continues down that path (as opposed to < 0) then at column PPN2 and so on performs a similar decision based off of the training data until it reaches a leaf at which the result is a 0, 1, or 2 in your case.

In reality, the model will be fit using other cutoffs determined by the training data and parameters you set. Additionally, not all of the columns will necessarily be used and they will not necessarily be in the order you see them. Furthermore with an extension like RF, you are going to fit thousands of trees and basically have the outcomes averaged to determine a final prediction (this helps with over fitting).

This link (an 100s of others) can point you to a decent starting point: https://www.r-bloggers.com/how-to-implement-random-forests-in-r/

This method usually performs very well in prediction but is not so great in interpretability. However, if you fit a single tree, there are ways to visualize it to get an idea of the underlying mechanics.

This link provides a pretty good overview of how to quickly train and visualize a tree: http://www.di.fc.ul.pt/~jpn/r/tree/tree.html

Hope this helps!

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you @Logan! I did not expect such a quick reply. To specify my first question: the channels are positions on the head where neural data was obtained. I would like to see if I can specify if some channels are better at capturing the difference in condition (category) and if so, to what extend they do (compared to other channels). The Random Forrest works out fine! (accuracy 80%) $\endgroup$ – QuickQuestion Jan 21 '20 at 23:09
  • $\begingroup$ Although working out fine, why would a Random Forrest be better option than a mixed regression model with subject ID as a random effect and category as a fixed effect? In order to see if the category can predict the neural data over all subjects but taking into account their individual('within')-variance. $\endgroup$ – QuickQuestion Jan 21 '20 at 23:29
  • $\begingroup$ Sorry for the delayed response! From what I understand, it may be difficult to determine which channels are best with such a small sample from each channel. As far as RF vs mixed regression: I wasn't suggesting RF thinking it was better but more along the lines of an alternative. I don't know much about mixed models, but I do believe that if you have a fixed variable it shouldn't be what you are trying to predict. Do you mean treat the channels as fixed? This could be a decent approach as well, but I wouldn't be able to tell which is best without running diagnostic tests on this specific case. $\endgroup$ – Logan Harris Jan 22 '20 at 13:40
  • $\begingroup$ No worries! Thanks for coming back. I have tried a lot since. I have performed a repeated measures ANOVA, which told me a significant difference was found comparing my 3 categories (I averaged over channels for this and used anova_test() and pairwise comparisons resulted in significant differences between all categories. Fine results to write my thesis, but still.. It would be nice to be able to specify where this difference is coming from (which you say is difficult due to sample size) OR if my data is 'strong enough' to predict. If possible, predictive power is what I'm looking for. $\endgroup$ – QuickQuestion Jan 22 '20 at 16:27
  • $\begingroup$ Yeah, I wasn't sure exactly what your end goal was so if it is a thesis I can reach out to a friend who may be able to give you a better input. I am starting my Ph.D. in biostats this upcoming fall so I'm far from an expert but my friend is a few years in.. The thing with your ANOVA is that it may show significant variation in your three categories but if that is what you are predicting then you can't leverage that in your prediction. As far as models go, other than a neural net, an RF will probably do the best for prediction accuracy, especially if you hone in the tuning parameters. $\endgroup$ – Logan Harris Jan 22 '20 at 23:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.