2
$\begingroup$

I'm trying to understand the "classic" iterative proportional fitting (IPF) algorithm.

Does it always assume that the variables being analyzed are independent? If the variables are independent, then can't we just compute each joint count from the product of two marginal counts?

For example, in the wikipedia example referenced above, are the two variables (gender and handedness) independent? The final joint counts are

45.24 = 100 * 52/100 * 87/100
41.76 = 100 * 48/100 * 87/100
6.76 = 100 * 52/100 * 13/100
6.24 = 100 * 48/100 * 13/100

so no iterative algorithm is necessary.

$\endgroup$
  • $\begingroup$ Are you still trying to understand the classic IPF algorithm? Did any of the three answers help? $\endgroup$ – conjugateprior Oct 23 '12 at 9:36
  • $\begingroup$ @ConjugatePrior: I understand things much better now. Thanks for your excellent answer! $\endgroup$ – Tyler Streeter Oct 25 '12 at 19:53
3
$\begingroup$

What you are seeing in the example is just a function of looking at a simple two-way table where there aren't any particularly interesting log linear models between independence and saturation.

In a three way table things may be a bit clearer. Take for example R's built-in data set HairEyeColor. This has dimensions Hair, Eye and Sex.

ftable(addmargins(HairEyeColor)) ## the data with all margins

        Sex Male Female Sum
Hair  Eye                      
Black Brown       32     36  68
      Blue        11      9  20
      Hazel       10      5  15
      Green        3      2   5
      Sum         56     52 108
Brown Brown       53     66 119
      Blue        50     34  84
      Hazel       25     29  54
      Green       15     14  29
      Sum        143    143 286
Red   Brown       10     16  26
      Blue        10      7  17
      Hazel        7      7  14
      Green        7      7  14
      Sum         34     37  71
Blond Brown        3      4   7
      Blue        30     64  94
      Hazel        5      5  10
      Green        8      8  16
      Sum         46     81 127
Sum   Brown       98    122 220
      Blue       101    114 215
      Hazel       47     46  93
      Green       33     31  64
      Sum        279    313 592

There are quite a few log linear models we might consider, all of which can be fitted by IPF and some of which need to be (or at least need an iterative procedure of some form). R's loglin function does fit in fact fit them all with IPF, whether or not a closed form is available. Let's consider two models.

IPF estimates cell values subject to marginal constraints. This corresponds to log linear models containing (or rather lacking) particular interaction terms. For example, suppose you got the idea that the connection between hair and eye colour was due entirely to (and thus conditionally independent given) sex. You'd then fit a model that contained a grand mean, main effects for Sex, Eye, and Hair, and two interaction terms Hair-Sex and Eye-Sex. This model can be written $[Hair,Sex][Eye,Sex]$ (or as the undirected graphical model $Hair-Sex-Eye$).

If you were to add Eye-Hair and Eye-Hair-Sex terms in the model then it would be saturated. If you were to take out the interaction terms then you'd be back to the independence model $[Hair][Sex][Eye]$.

Here's the conditional independence model in R:

mod1 <- loglin(HairEyeColor, list(c('Eye', 'Sex'), c('Hair', 'Sex')), 
               fit=TRUE)

Looking at its fitted values shows which margins are recovered and which not:

ftable(addmargins(mod1$fit))

            Sex       Male     Female        Sum
Hair  Eye                                       
Black Brown      19.670251  20.268371  39.938622
      Blue       20.272401  18.939297  39.211699
      Hazel       9.433692   7.642173  17.075864
      Green       6.623656   5.150160  11.773816
      Sum        56.000000  52.000000 108.000000
Brown Brown      50.229391  55.738019 105.967410
      Blue       51.767025  52.083067 103.850092
      Hazel      24.089606  21.015974  45.105580
      Green      16.913978  14.162939  31.076918
      Sum       143.000000 143.000000 286.000000
Red   Brown      11.942652  14.421725  26.364378
      Blue       12.308244  13.476038  25.784282
      Hazel       5.727599   5.437700  11.165298
      Green       4.021505   3.664537   7.686042
      Sum        34.000000  37.000000  71.000000
Blond Brown      16.157706  31.571885  47.729591
      Blue       16.652330  29.501597  46.153927
      Hazel       7.749104  11.904153  19.653257
      Green       5.440860   8.022364  13.463224
      Sum        46.000000  81.000000 127.000000
Sum   Brown      98.000000 122.000000 220.000000
      Blue      101.000000 114.000000 215.000000
      Hazel      47.000000  46.000000  93.000000
      Green      33.000000  31.000000  64.000000
      Sum       279.000000 313.000000 592.000000

Notice that all the margins except the one in the right hand column are the same as in the original data.

An alternative model that captures this margin while still being neither the pure independence model nor saturated is $[Hair,Sex][Eye,Sex][Eye,Hair]$:

mod2 <- loglin(HairEyeColor, list(c('Eye', 'Sex'), c('Hair', 'Sex'), 
               c('Hair','Eye')), fit=TRUE)

This model assumes that all odds ratios between any two variables are the same at each level of the third ('homogeneous association'). This takes six IPF iterations to fit and has no closed form estimates.

ftable(addmargins(mod2$fit))

            Sex       Male     Female        Sum
Hair  Eye                                       
Black Brown      32.793454  35.206546  68.000000
      Blue       11.743353   8.256647  20.000000
      Hazel       8.444542   6.555458  15.000000
      Green       3.018490   1.981510   5.000000
      Sum        55.999839  52.000161 108.000000
Brown Brown      52.524536  66.475464 119.000000
      Blue       45.930572  38.069428  84.000000
      Hazel      28.196301  25.803699  54.000000
      Green      16.348411  12.651589  29.000000
      Sum       142.999820 143.000180 286.000000
Red   Brown      10.760594  15.239406  26.000000
      Blue        8.819781   8.180219  17.000000
      Hazel       6.916815   7.083185  14.000000
      Green       7.502806   6.497194  14.000000
      Sum        33.999996  37.000004  71.000000
Blond Brown       1.926579   5.073421   7.000000
      Blue       34.500428  59.499572  94.000000
      Hazel       3.443330   6.556670  10.000000
      Green       6.129759   9.870241  16.000000
      Sum        46.000097  80.999903 127.000000
Sum   Brown      98.005164 121.994836 220.000000
      Blue      100.994134 114.005866 215.000000
      Hazel      47.000988  45.999012  93.000000
      Green      32.999465  31.000535  64.000000
      Sum       278.999751 313.000249 592.000000

As expected, its fitted values now also recover the third column margins that mod1 did not because of the extra interaction term.

As @gjabel notes, Agresti 2002 contains a nice clear discussion. Hope that helps.

$\endgroup$
  • $\begingroup$ Thanks for walking through an example... the 3-way example really helps clarify things. $\endgroup$ – Tyler Streeter Oct 23 '12 at 19:22
5
$\begingroup$

A variation of the IPF is frequently used in survey statistics to adjust for the error in the data, where it is known as the raking ratio adjustment. Initially it was proposed to deal with sampling error (the difference between the true proportions and the observed proportions due to the fact that not everybody is being observed), and these days it is used more to correct for non-sampling error (not all values are observed because of non-response). Starting from the sample values, the weights are adjusted so that a given margin corresponds to the known population value, until convergence. This algorithm solves a certain optimization problem of finding the joint distribution that is closest to that of the sample, yet satisfies the marginal distributions. See http://www.citeulike.org/user/ctacmo/article/10383730/.

Lacking any additional information about the joint distribution, the joint independence is, in a way, the most natural assumption you could possibly make.

$\endgroup$
1
$\begingroup$

You can use IPF to fit any log-linear models. If your model is relatively simple and has no interactions terms, then no iterations are required. For example the model being fitted in the wikipedia article is: $$\log m_{ij} = \beta_i + \beta_j .$$ However, if you are fitting a more complex model, perhaps with interactions over contingency tables with more than two dimensions then you will need to to iterate to find the mle. Agresti's chapter on log-linear models (in his Categorical Data Analysis book) explains which log-linear models can be solved directly, and which need an iterative solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.