in order to compare human action sequences and computer modeled predictions for action sequences I use a similarity measure for these sequences. All similarities can have a value between 0 (terrible prediction) and 1 (perfect prediction). In total I have 24 humans with 36 tasks and the respective action sequences, model predictions and similarity values.
I am able to vary the number of parameters that influence the predictions. With more parameters I receive a model that is able to predict action sequences that are more similar to the human action sequences. The perfect model would create for each human a similarity of 1.
For comparing different models (with a different number of parameters) I like to penalize the increasing number of parameters, so I found the Bayesian Information Criterion (BIC) that uses a log likelihood function and the number of parameters.
For example my first model with 6 parameters creates a mean similarity of 0.4 and my better model with 8 parameters creates a mean similarity of 0.5.
Is it possible to use the BIC for my domain or are there other ways to justify my increased number of parameters? If I can use the BIC, how can I formulate a likelihood function that would be needed for the BIC or similar procedures?
If you need more information, just ask.