# Number of parameters in Markov model

I want to use BIC for HMM model selection:

BIC = -2*logLike + num_of_params * log(num_of_data)


So how do I count the number of parameters in the HMM model. Consider a simple 2-state HMM, where we have the following data:

data = [1 2 1 1 2 2 2 1 2 3 3 2 3 2 1 2 2 3 4 5 5 3 3 2 6 6 5 6 4 3 4 4 4 4 4 4 3 3 2 2];
model = hmmFit(data, 2, 'discrete');
model.pi = 0.6661    0.3339;
model.A =
0.8849    0.1151
0.1201    0.8799
model.emission.T =
0.2355    0.5232    0.2259    0.0052    0.0049    0.0053
0.0053    0.0449    0.2204    0.4135    0.1582    0.1578
logLike = hmmLogprob(model,data);
logLike =  -55.8382


So I think:

Nparams = size(model.A,2)*(size(model.A,2)-1) +
size(model.pi,2)-1) +
size(model.emission.T,1)*(size(model.emission.T,2)-1)
Nparams = 13


So at the end we have:

BIC = -2*logLike + num_of_params*log(length(x))
BIC = 159.6319


I've found a solution where the formula for num_of_params (for simple Markov model) looks like:

Nparams = Num_of_states*(Num_of_States-1) - Nbzeros_in_transition_matrix


So what's the right solution? Do I have to take into account some zero probabilities in transition or emission matrices?

====Updated since 07.15.2011====

I think I can provide some clarification on the impact of data dimension (using “Gaussian mixture distribution” example)

X is an n-by-d matrix where (n-rows correspond to observations; d-columns correspond to variables (Ndimensions).

X=[3,17 3,43
1,69 2,94
3,92 5,04
1,65 1,79
1,59 3,92
2,53 3,73
2,26 3,60
3,87 5,01
3,71 4,83
1,89 3,30 ];
[n d] = size(X);
n = 10; d =2;


The model will have the following number of parameters for GMM:

nParam = (k_mixtures – 1) + (k_mixtures * NDimensions ) + k_mixtures * Ndimensions  %for daigonal covariance matrices
nParam = (k_mixtures – 1) + (k_mixtures * NDimensions ) + k_mixtures * NDimensions * (NDimensions+1)/2; %for full covariance matrices


If we treat X as 1-dimentional data, than we have num_of_data = (n*d), so for the 2-dimentional data we have num_of_data = n.

2-dimentional data: nParam = 11 ; logLike = -11.8197; BIC = 1.689

1-dimentional data: nParam = 5 ; logLike = -24.8753; BIC = -34.7720

I have a very little practice with HMM. Is it normal to have HMM with (5000, 6000 and more parameters)?

-
do you have a justification for using BIC? It can give horribly wrong results if not with the appropriate assumptions. – suncoolsu Jul 15 '11 at 12:07
@suncoolsu , What do you mean about justification? I've found some examples on K-clusters (GMM models) selection based on BIC scoring. Probably I've provided wrong example with comparing two models with different input data (dimensions). – Sergey Jul 18 '11 at 8:41
I meant using BIC only if the assumption, the true model is in the model space, is justified. May be it is justified in your case. I agree with you that people use BIC like AIC, but both are very different things! – suncoolsu Jul 18 '11 at 8:44
Hi, this is a late comment and hope you are still active, but what is the best way to get the number of parameters in a model? – masfenix Sep 29 '14 at 23:02

Do I have to take data dimension into account? What if size(data) will be 2x100 – Sergey Jul 12 '11 at 4:52