0
$\begingroup$

I am going through two materials regarding Conditional Random Fields. The first one is this (referred to as [1]) material by Charles Sutton and Andrew McCallum and the second one is this (referred to as [2]). In [1], on page 79, it is stated that feature functions can be re-expressed in order to share a single set of weights across all the classes in logistic regression, rather than using one weight vector per class. According to [1], if we have two classes $c_{1}$ and $c_{1}$, each with feature $f_1(x,c)$, the feature will be re-expressed like $f_{c^{'},1}(x,c) =1\{c^{'}= c\}~f_1(x,c)$.

I do not clearly understand how it enables weight sharing across classes. Does it mean that, for the above example, we now keep two features for each class instead of one like - $f_{c_{1},1}(x,c) =1\{c_{1}= c\}~f_1(x,c_{1})$ and $f_{c_{2},1}(x,c) =1\{c_{2}= c\}~f_1(x,c_{2})$, where each one will trigger (have value 1) for a single class and will be zero for others? Even if we assume this to be correct, a feature value can be non-zero for more than one class (where a class is assumed to be a single sequence configuration) in CRF. Hence, it seems like such weight sharing scheme is not necessarily equivalent to different weights for different classes in multinomial logistic regression.

The second question is, why such weight sharing is required for CRF? The same notation for such shared weights is used in both [1] and [2]. Why single weight vector per class is not used like multinomial/ sigmoid logistic regression?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.