0
$\begingroup$

I'm quite new to this and wondering if you can help. I'm looking to create an SVM to determine whether neural data from several electrodes can predict behavioral performance on a task. Does the following sound like a reasonable workflow to perform this? I'll be using Matlab, just due to familiarity. I may have as few as 66 trials. I really appreciate any help and advice:

Get data into samples (e.g. trials) x features/predictors

Add column for classification (probably first column)

Select, for example, 80% of dataset to become testing/tuning data

Create model and tune hyperparameters (using nested cross-validation perhaps) using only this 80% of dataset; ensure model is implementing standardization and stratification of classes

Apply tuned model to final 20% test data

Shuffle data (rows) n times (e.g. 1000) to randomise 20% test data and then predict again each time - don't retrain model, just predict using tuned model.

Average predictions and get distribution across shuffled dataset - this is the performance of the model and therefore gives me a description of how well the neural data predict the behavioral response

I originally used the scripted output from classificationLearner in MATLAB with k-fold CV to develop the model based on all the data, and used the validationAccuracy output as my performance measure, but the more I think about it I'm not sure this approach was accurate. Thank you!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.