I have seen these two terms used often. I can not really tell the difference between them they seem Identical to me. Can someone point me to a resource or give a simple explanation of what the difference is if any?

By log-linear I mean a model where the conditional probability of a random variable $y$ given some feature vector $f(x,y)$. There parameters of the model are weights $\theta$.

$$ p(y \mid x) = \frac{\exp(\theta \cdot f(x,y))}{\sum_{y'} \exp(\theta \cdot f(x,y'))} $$

  • 2
    $\begingroup$ These terms mean different things in different contexts. Statistics has developed as its own discipline but also within other disciplines (think econometrics, psychometerics, etc.). As a consequence there are many examples of the same name that has been used for different things and the same thing that has been given different names. Can you give us the context within which you found those terms? $\endgroup$ – Maarten Buis Jun 16 '15 at 7:37
  • $\begingroup$ @MaartenBuis thanks for the comment, I added an example of what I mean by log-linear. $\endgroup$ – A.D Jun 16 '15 at 14:34

A bilinear function (see https://en.wikipedia.org/wiki/Bilinear_map ) is a function which is linear in each variable when all other variables are fixed. For instance, f(x,y) = x * y.

Your p(y | x) is log-linear if f(x,y) is linear in x and y. It is log-bilinear if f(x,y) is bilinear in x and y.

See slide 9 of http://www.cs.utoronto.ca/~hinton/csc2535/notes/hlbl.pdf and slides 17, 23, and 24 of https://piotrmirowski.files.wordpress.com/2014/06/piotrmirowski_2014_wordembeddings.pdf as reference.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.