Let's consider, for instance, a table like this one, with two independent categorical variables:

x <- Titanic[,,2,1]

I want to model this contingency table using log-linear fitting. If I assume independence of the row and column variable, I can write:

$log~\lambda_{ij}= \eta + \theta_i + \delta_j$

Which in R is:

loglin(x, c(1,2), fit=T)

This model considers that the value in cell $ij$ comes from a random Poisson variable with parameter $\lambda_{ij}$. This parameter depends on a constant, a row component and a column component. The matrix "fit" from the output of the R function contains these fitted values.

On the other hand, I can also compute the expected values for this contingency tabe:

$E_{ij} = Np_ip_j$

where $p_i$ is the fraction of elements being in row $i$, $p_j$ is the fraction of elements in column $j$, and $N$ the total number of elements in the table. In R:


Well, it turns out that:

$E_{ij}= \lambda_{ij}$

This observation seems intuitively true in general, am I wrong? Could anybody provide me an explanation?


Yes, this is a structural truth about linear models scored on their training data.

The structure equation satisfied by a linear model is

$$ X^t X \beta = X^t y $$ Here $\beta$ are the fit model coefficients and $y$ is your log transformed response. Note that $X \beta$ are the model predictions, so we can re-write this relation as

$$ X^t p = X^t y $$

where $p$ is the prediction vector.

Now take some categorical predictor in your model, and find a column in $X$ corresponding to one of its levels. This column will become a row in $X^t$, say it's the i'th row. This row will consist of zeros and ones, encoding the level of the categorical predictor. Now extract the equation corresponding to this row from the structure equation

$$ \sum_j X^t_{ij} p_j = \sum_j X^t_{ij} y_j $$

Many of the $X^t_{ij}$ terms are zero, the remaining are ones, so we can rewrite this equation as

$$ \sum_{X^t_{ij} = 1} p_j = \sum_{X^t_{ij} = 1} y_j $$

which is a restatement of your observed relationship between contingency tables.

  • $\begingroup$ I'm not sure this is an answer to the question (or either I don't understand your reasoning). I don't have binary X matrix, it is a contingency matrix. Furthermore, the predicted value p is different from the actually observed. What I am asking is rather if $\lambda_{ij}$, i.e. the predicted value from the log-linear model, is always equal to the expected value for that cell (e.g. $\sum row_i \sum col_j / N$) $\endgroup$ – retrot Jun 19 '15 at 11:59
  • $\begingroup$ My $X$ matrix would be your design matrix in your linear model. I may have misunderstood you as well. Could you add details about your model, the nature of the variables in your model, and how you construct the contingency table? If I can, ill update my answer. I'm not sure what a predicted $p$ value is, could you define that term? $\endgroup$ – Matthew Drury Jun 19 '15 at 15:04
  • $\begingroup$ Sorry for the delay, I edited my question, I hope is more clear now. Thanks for the feedback anyway. $\endgroup$ – retrot Jun 24 '15 at 9:08
  • $\begingroup$ @retrot Thanks! I'll try to take a look after work. $\endgroup$ – Matthew Drury Jun 24 '15 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.