Machine Learning application to neuroscientific data I have the following question, and it is (for me) an open research question. I really don't have a feeling if it is a difficult question or a trivial one, getting a better feeling for approaches I might take by your feedback would be much appreciated. I won't explain what I have done so far to not make it too complicated/biased, but would be happy to give more details if anything is unclear. 
This is the behavioral experiment: A subject receives simultaneously vibration to a finger from the left hand as well as to a finger of the right hand. What the participant has to do is to indicate where the stronger*  stimulation is (*or the weaker stimulation is, these two questions are randomized, and it is randomized as well which side receives the stronger stimulation). 
There are three difficulty levels: The two stimulations the subject receives can have a very different intensity, s.t. subject can answer the question 100% correctly. 
There is a medium level where participants can identify the stronger/weaker stimulation with 75% accuracy, and a difficulty level where the two stimuli are so similar that subjects accuracy gets close to (theoretical) chance level.
We do this experiment in the fMRI scanner. The goal is to predict from this neuroimaging data, how certain participants are (100%, 75%, 50%?). 
The question sounds rather simple, however I have some quite basic questions regarding what to use as classes for the classification, what as labels (subjects perception of where they are being stimulated more strongly, or the effective side, or something completely else...?).
 A: They question is a little vague, but I'll have a try :
The three main conditions of your experiment are 100% ("certain", or easy), 75% ("neither", or medium), or 50% ("uncertain", or hard).
A straightforward way would be to follow the experimental design and perform a multi-class classification trying to predict whether a given neuroimage belongs to one of three classes (see: difference between "classification" and "labeling" problems):


*

*0: "Uncertain" 50%

*1: "Neither" 75%

*2: "Certain" 100%


I do not know which algorithms/software you are planning to use, but most packages will require you to simply code the three conditions as integers (e.g. 0,1,2, as in this example) and use these as prediction targets.
Another way to approach your goal would be as a regression problem: You could try and predict the observed accuracy of participants (...or even certainty ratings). I assume that your individual participants deviated from the desired hit-rates of 50-75-100. If so, you can take the observed accuracy (or some other psychophysical performance measure like d') and define this as the prediction target of a (ridge-, LASSO-, elastic-net- etc.) regression. However, the experiment was not really designed for this approach, so maybe stick with the first one.
Generally, the experiment above will not be enough to achieve your goal to build a predictor of participants' "certainty". The main problem will be to argue that your prediction really captures "certainty". What you  manipulated experimentally was "task difficulty in a somatosensory discrimination task", or "side imbalances in somatosensation". To really infer on "certainty", you would need a much larger variety of decision-making tasks. Further, participant ratings of certainty will  be necessary to validate your approach.
