I'm quite new to this and wondering if you can help. I'm looking to create an SVM to determine whether neural data from several electrodes can predict behavioral performance on a task. Does the following sound like a reasonable workflow to perform this? I'll be using Matlab, just due to familiarity. I may have as few as 66 trials. I really appreciate any help and advice:

Get data into samples (e.g. trials) x features/predictors

Add column for classification (probably first column)

Select, for example, 80% of dataset to become testing/tuning data

Create model and tune hyperparameters (using nested cross-validation perhaps) using only this 80% of dataset; ensure model is implementing standardization and stratification of classes

Apply tuned model to final 20% test data

Shuffle data (rows) n times (e.g. 1000) to randomise 20% test data and then predict again each time - don't retrain model, just predict using tuned model.

Average predictions and get distribution across shuffled dataset - this is the performance of the model and therefore gives me a description of how well the neural data predict the behavioral response

I originally used the scripted output from classificationLearner in MATLAB with k-fold CV to develop the model based on all the data, and used the validationAccuracy output as my performance measure, but the more I think about it I'm not sure this approach was accurate. Thank you!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.