# log-linear vs log-bilinear

I have seen these two terms used often. I can not really tell the difference between them they seem Identical to me. Can someone point me to a resource or give a simple explanation of what the difference is if any?

By log-linear I mean a model where the conditional probability of a random variable $y$ given some feature vector $f(x,y)$. There parameters of the model are weights $\theta$.

$$p(y \mid x) = \frac{\exp(\theta \cdot f(x,y))}{\sum_{y'} \exp(\theta \cdot f(x,y'))}$$

• These terms mean different things in different contexts. Statistics has developed as its own discipline but also within other disciplines (think econometrics, psychometerics, etc.). As a consequence there are many examples of the same name that has been used for different things and the same thing that has been given different names. Can you give us the context within which you found those terms? Commented Jun 16, 2015 at 7:37
• @MaartenBuis thanks for the comment, I added an example of what I mean by log-linear.
– A.D
Commented Jun 16, 2015 at 14:34

## 2 Answers

A bilinear function (see https://en.wikipedia.org/wiki/Bilinear_map) is a function that is linear in each variable when all other variables are fixed. For instance, $$f(x, y) = x y$$.

Your $$p(y \mid x)$$ is log-linear if $$f(x, y)$$ is linear in $$x$$ and $$y$$. It is log-bilinear if $$f(x, y)$$ is bilinear in $$x$$ and $$y$$.

See slide 9 of http://www.cs.utoronto.ca/~hinton/csc2535/notes/hlbl.pdf and slides 17, 23, and 24 of https://piotrmirowski.files.wordpress.com/2014/06/piotrmirowski_2014_wordembeddings.pdf as reference.

The textbook categorical data analysis treatment, say Agresti (2002), of a $$N\times D$$ contingency table $$C$$ with components $$C_{ij}$$ involves three log linear model \begin{align*} \log E[C_{ij}] =& \lambda & \text{(constant rate)}\\ \log E[C_{ij}] =& \lambda + \lambda^R_i + \lambda^C_j & \text{(independence)}\\ \log E[C_{ij}] =& \lambda + \lambda^R_i + \lambda^C_j + \lambda^{RC}_{ij} & \text{(saturated)} \end{align*} where $$\lambda$$ is scalar, $$\lambda^R$$ is length $$N$$, $$\lambda^C$$ is length $$D$$, and $$\lambda^{RC}$$ is an $$N\times D$$ matrix of of interaction terms. Typical identification constraints include that $$\lambda^R$$ and $$\lambda^C$$ have mean 0.

Log bilinear models have complexity between the independence and saturated models (but require a different fitting process due to the multiplicative term). The simplest log bilinear model would be \begin{align*} \log E[C_{ij}] & = \lambda + \lambda^R_i + \lambda^C_j + \mu_i \nu_j & \text{(RC model* / log-bilinear)} \end{align*} where $$\mu$$ is length $$N$$, $$\nu$$ is length $$D$$. (*Goodman's original formulation absorbs $$\lambda$$ into another parameter). Extra identification constraints are required here, e.g. Z-scored $$\mu$$ and mean zero $$\nu$$.

The standard treatment of these is from Goodman (1979), who called this the RC association model, but as far as I can see this is the generic log bilinear structure.

Increasing the number of multiplicative terms to, say M, moves the flexibility of the model towards the saturated model, and are referred to as RC(M) models, although only the RC model seems to called log bilinear.