1
$\begingroup$

in order to compare human action sequences and computer modeled predictions for action sequences I use a similarity measure for these sequences. All similarities can have a value between 0 (terrible prediction) and 1 (perfect prediction). In total I have 24 humans with 36 tasks and the respective action sequences, model predictions and similarity values.

I am able to vary the number of parameters that influence the predictions. With more parameters I receive a model that is able to predict action sequences that are more similar to the human action sequences. The perfect model would create for each human a similarity of 1.

For comparing different models (with a different number of parameters) I like to penalize the increasing number of parameters, so I found the Bayesian Information Criterion (BIC) that uses a log likelihood function and the number of parameters.

For example my first model with 6 parameters creates a mean similarity of 0.4 and my better model with 8 parameters creates a mean similarity of 0.5.

Is it possible to use the BIC for my domain or are there other ways to justify my increased number of parameters? If I can use the BIC, how can I formulate a likelihood function that would be needed for the BIC or similar procedures?

If you need more information, just ask.

$\endgroup$
  • $\begingroup$ How many separate actions in each sequence? You should also state the models you did use, with algebraic details. $\endgroup$ – kjetil b halvorsen Sep 14 '15 at 14:18
  • $\begingroup$ The number of actions in each sequence varies. As a result I used a relative similarity value with the Smith-Waterman algorithm: Rel. sim value: SW(a,b) / max(length(a), length(b)) The models I used were means-ends analysis models that had to solve a planning task. The resulting sequences are the actions to reach the next state. The models differed in the number of strategies that are used to determine the next action. $\endgroup$ – B. Rendan Sep 14 '15 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.