I am reading Daphne Koller's book on probabilistic graphical models under the topic of conditional random fields. One of the advantages in using CRF is that we can avoid modelling the correlations between random variables in $X$. In the case of naive bayes model, the assumption when modelling the joint distribution is that the features $x_i$ are conditionally independent given the class $c$. Below is the quote from the book.

The fact that we avoid encoding the distribution over the variables in X is one of the main strengths of the CRF representation. This flexibility allows us to incorporate into the model a rich set of observed variables whose dependencies may be quite complex or even poorly understood. It also allows us to include continuous variables whose distribution may not have a simple parametric form. This flexibility allows us to use domain knowledge in order to define a rich set of features characterizing our domain, without worrying about modeling their joint distribution

My confusion is that the coursera lecture, the logistic model is also defined from the joint distribution.

$$ \phi_i(X_i,Y=1) = exp(w_ix_i)\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \phi_i(X_i,Y=0) =1 \\ \tilde{P}_{\Phi}(\textbf{X},Y=1) = exp\{\sum_i(w_iX_i)\}\,\,\,\, \tilde{P}_{\Phi}(\textbf{X},Y=0)=1\\ $$

where $\phi_1(x_i,y)$ is a factor over feature $x_i$ and label $y$.

I don't get how the correlation of features can be avoided when using a logistic regression since the graph structure is similar to a naive baye's model (when computing the factor product to find $\tilde{P}_{\Phi}(X_i,Y=1)$ etc.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.