9
$\begingroup$

I have two logistic regression models in R made with glm(). They both use the same variables, but were made using different subsets of a matrix. Is there an easy way to get an average model which gives the means of the coefficients and then use this with the predict() function?

[ sorry if this type of question should be posted on a programming site let me know and I'll post it there ]

Thanks

$\endgroup$
  • 1
    $\begingroup$ You might find some useful information in the related thread at stats.stackexchange.com/q/8502/919 . $\endgroup$ – whuber Apr 1 '11 at 3:04
  • $\begingroup$ In R, the caret package has some cool features for combining models. $\endgroup$ – screechOwl Dec 5 '11 at 16:01
2
$\begingroup$

Do you want to take the average of the predicted probabilities, or the average of the coefficients? They will give different results, because a logistic regression involves a nonlinear transform of the linear predictor.

A function to do either would be something like this. Set avg to "prob" to get the former, or something else for the latter.

pred_comb <- function(mod1, mod2, dat, avg="prob", ...)
{
    xb1 <- predict(mod1, dat, type="link", ...)
    xb2 <- predict(mod2, dat, type="link", ...)
    if(avg == "prob")
        (plogis(xb1) + plogis(xb2))/2
    else plogis((xb1 + xb2)/2)
}
$\endgroup$
  • $\begingroup$ @Hong Ooi: Thanks! This code is really useful, and will do the trick, but what I was mainly aiming for was a function which would give a new R model which I could then put into predict() later on in the workflow. However, I think your function is pretty elegant, and like the way you can set it to give probabilities. $\endgroup$ – Andrew Apr 1 '11 at 0:54
  • $\begingroup$ @Andrew you can take advantage of R's object-oriented programming for this. Put your two models into a list, and give it a class, say glm_2. Call the above function predict.glm_2 and you can then use predict() on your object as required. $\endgroup$ – Hong Ooi Apr 1 '11 at 1:04
  • 2
    $\begingroup$ Why is averaging the coefficients appropriate? If the two datasets have different sizes, then surely at least some form of weighted average should be used. $\endgroup$ – whuber Apr 1 '11 at 3:05
  • $\begingroup$ @ Hong - thanks, I'll try this out. @whuber - thanks for pointing this out. If my two datasets are the same size, is weighting an issue (I hadn't thought of this before). If they are different do I just weight by the ratio of the sizes (e.g. the if one is twice as big, do I give it twice the weight)? $\endgroup$ – Andrew Apr 1 '11 at 10:05
  • $\begingroup$ @Andrew It is correct to weight by data count only if the values of the independent variables are the same in both subsets. For a better approach (which is also theoretically valid), see the thread I referenced in a comment to your question. $\endgroup$ – whuber Apr 1 '11 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.