0
$\begingroup$

I have the following question, and it is (for me) an open research question. I really don't have a feeling if it is a difficult question or a trivial one, getting a better feeling for approaches I might take by your feedback would be much appreciated. I won't explain what I have done so far to not make it too complicated/biased, but would be happy to give more details if anything is unclear.

This is the behavioral experiment: A subject receives simultaneously vibration to a finger from the left hand as well as to a finger of the right hand. What the participant has to do is to indicate where the stronger* stimulation is (*or the weaker stimulation is, these two questions are randomized, and it is randomized as well which side receives the stronger stimulation).

There are three difficulty levels: The two stimulations the subject receives can have a very different intensity, s.t. subject can answer the question 100% correctly.

There is a medium level where participants can identify the stronger/weaker stimulation with 75% accuracy, and a difficulty level where the two stimuli are so similar that subjects accuracy gets close to (theoretical) chance level.

We do this experiment in the fMRI scanner. The goal is to predict from this neuroimaging data, how certain participants are (100%, 75%, 50%?).

The question sounds rather simple, however I have some quite basic questions regarding what to use as classes for the classification, what as labels (subjects perception of where they are being stimulated more strongly, or the effective side, or something completely else...?).

Improve this question
$\endgroup$
2
$\begingroup$

They question is a little vague, but I'll have a try :

The three main conditions of your experiment are 100% ("certain", or easy), 75% ("neither", or medium), or 50% ("uncertain", or hard).

A straightforward way would be to follow the experimental design and perform a multi-class classification trying to predict whether a given neuroimage belongs to one of three classes (see: difference between "classification" and "labeling" problems):

  • 0: "Uncertain" 50%
  • 1: "Neither" 75%
  • 2: "Certain" 100%

I do not know which algorithms/software you are planning to use, but most packages will require you to simply code the three conditions as integers (e.g. 0,1,2, as in this example) and use these as prediction targets.

Another way to approach your goal would be as a regression problem: You could try and predict the observed accuracy of participants (...or even certainty ratings). I assume that your individual participants deviated from the desired hit-rates of 50-75-100. If so, you can take the observed accuracy (or some other psychophysical performance measure like d') and define this as the prediction target of a (ridge-, LASSO-, elastic-net- etc.) regression. However, the experiment was not really designed for this approach, so maybe stick with the first one.

Generally, the experiment above will not be enough to achieve your goal to build a predictor of participants' "certainty". The main problem will be to argue that your prediction really captures "certainty". What you manipulated experimentally was "task difficulty in a somatosensory discrimination task", or "side imbalances in somatosensation". To really infer on "certainty", you would need a much larger variety of decision-making tasks. Further, participant ratings of certainty will be necessary to validate your approach.

Improve this answer
$\endgroup$
  • $\begingroup$ Many thanks for the comment. Agreed it is not really certainty in that sense that we are capturing here with this task. The idea is however this: If once subjects can distinguish very well between the stimuli and once not at all (chance level), some information in the brain must reflect - hence could we train a classifier which would have the same performance as the subjects? And how would we train it? ...> $\endgroup$ – Pugl Feb 10 '17 at 11:18
  • $\begingroup$ ...>For instance I did a binary classification for each of the three conditons separately, hoping that in the easy case, a classifier can perfectly distinguish between trials where the right hand was stimulated more strongly than the left hand was, and that in the difficult case the classifier's accuracy would also be at around chance level. This would give us a map from the brain activity to a performance assessment without having to test the subject behaviorally. Unfortunately, so far this approach has not led to the desired results, and I don't know why... $\endgroup$ – Pugl Feb 10 '17 at 11:19
  • 1
    $\begingroup$ Identifying whether the left or the right hand was stimulated more strongly is a completely different (but not uninteresting) task. In this case, you'd train the classifier with labels "left" and "right". Then you could compare performance of this left-right classifier with participants' actual performance. Given, that the classifier preforms as good as the participants you could then use it as a proxy for "discriminatory information" in later studies. $\endgroup$ – mzunhammer Feb 10 '17 at 11:51
  • 1
    $\begingroup$ Ok then you do find this a sensible approach in fact. Tx! $\endgroup$ – Pugl Feb 10 '17 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.