In neuroscience it is very common to measure the reaction time (RT) of subjects. Based on the RT conclusions can be made about personal working memory capability, IQ etc.

So I have such data, which came from some neuroscience experiment. In this data I have 180 subjects from two groups (90 subjects in each group) for simplicity let say boys and girls, and I have a set of 500 RTs for each subject. The difference between the mean RT of each group is statistically significant (t-test). Now, I would like to construct a classifier which will learn the differences between those two groups. I want the classifier to classify new subjects only based on their RT - and I'm talking specifically about classification of RT.

(A) I'm searching for dimensionality-reduction method which would fit to this case. ( I have tried PCA and it doesn't work well ). Not sure if PCA is the right thing here as there is no difference between the 100th and the 200th RT. Each RT is independent and there is no specific ordering - so there is not any co-variance here that PCA can capture. (right ?)

(B) Which classifier is recommended in such case ?

Does anyone aware of such kind of work in general and specific for RT ? Any information on this would be appreciated.

  • $\begingroup$ This is a rephrase of a question I've posted here yesterday. (and excuse me for my not so good English) $\endgroup$ – Dov Feb 14 '12 at 7:35
  • $\begingroup$ So what do the 500 RTs mean? Are they repetitions of the same experiment? or are they executions of different experiments? Why does it "stand on its own" and "order has no meaning"? $\endgroup$ – carlosdc Feb 14 '12 at 8:31
  • $\begingroup$ Looks like there are still questions about what the experiment is. I presume in this experiment 90 "boys" and 90 "girls" were asked to give a response to 500 different stimuli (each) and the 500 stimuli were the same for the "boys" and the "girls" and their reaction time (RT) was recorded. So you have 180 samples with 500 features and one output. 500 features/dimensions is not that large, so are you trying to use PCA to identify the significant stimuli that you can use to discriminate between "boy" and "girl" outputs? $\endgroup$ – Maybe Feb 14 '12 at 18:24
  • $\begingroup$ Yes, maybe PCA wasn't the right thing here - I must say that I didn't know that 500 is 'not that large':) I have use PCA only for dimension-reduction. As I don't believe that among 500 button presses (RT) there is a one that will be significant different from the others. $\endgroup$ – Dov Feb 14 '12 at 18:40

A simple discriminative classifier should train in seconds and generalize well after tuning l1 and l2 regularization parameters on a dataset of this size. There is no need to do dimensionality reduction.

If you still needed to do dimensionality reduction for whatever reason, you could use random projections, independent components analysis, autoencoders, or any number of nonlinear techniques which work well in the absence of obvious linear relationships on the data.

  • 1
    $\begingroup$ It should be noted that L1 and L2 regularizations are approaches to dimensionality reduction. But a user might not know it. If any feature weight is reduced to 0, the dimension is effectively removed from the model. It's also fair to note that L2 regularization is pretty bad at this (i.e. setting a weight to exactly zero), compared to L1. (Hi Alexandre) $\endgroup$ – Andrew Rosenberg Feb 14 '12 at 12:53
  • $\begingroup$ First thanks for the answer. Second, Do you think that methods such "bag of words" can also be useful in this case ? $\endgroup$ – Dov Feb 14 '12 at 14:07

This description is closer to OK, but still you need to describe a lot of things in more detail.

Since you want to classify, it seems like what you want is LDA (Linear Discriminant Analysis) more than PCA. You want to "dimensionality reduction", possibly because you need to be able to describe the rule you obtain, but more importantly don't forget that you want something that helps you classify boys from girls.

The most important key step is that you will have to think of a sensible representation for your data that helps achieve this goal. Depending on this:

  • what do the 500 RTs mean? Are they repetitions of the same experiment? or
  • are they executions of different experiments? Why does it "stand on its own" and "order has no meaning"?

the representation would be significantly different.

Also when you say PCA does not work well, what exactly do you mean by that? It could be many things: does it give unacceptable accuracy on new data? or does it work reasonably well, just not as well as you had hoped?

What you say in your question (A), is not true you may be seeing the effects of a poor representation for your data.

  • $\begingroup$ yes you right my goal is classification but I thought I need to do some "dimensionality reduction" before the LDA because of the high dimension I have - 500 RT for each subject. $\endgroup$ – Dov Feb 14 '12 at 9:11
  • $\begingroup$ "stand on its own" - when you have many features for the same subject PCA assume ( if I understand it correctly..) that each feature has its own meaning (e.g. height, weight, size of shoes, age, sex, etc' etc' ) . where PCA seek to capture the variance along those features (i.e. the PCs) but here I have only one feature (the RT) with many instances (500) of the same feature $\endgroup$ – Dov Feb 14 '12 at 9:16
  • $\begingroup$ "PCA does not work well" please ignore this comment it not so important. the main thing here that I do not know how to tackle this classification problem. $\endgroup$ – Dov Feb 14 '12 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.