11
$\begingroup$

I'm looking at a few logistic regression issues. ("regular" and "conditional").

Ideally, I'd like to weight each of the input cases so that the glm will focus more on predicting the higher weighted cases correctly at the expense of possibly misclassifying the lower weighted cases.

Surely this has been done before. Can anyone point me toward some relevant literature (Or possibly suggest a modified likelihood function.)

Thanks!

$\endgroup$
9
  • 1
    $\begingroup$ You are assuming that classification is the goal, as opposed to prediction. For optimum estimation of probabilities you don't need to re-weight anything. "False negatives" and "false positives" only occur with forced choices, and usually no one is forcing a pure binary choice. $\endgroup$ Commented Aug 16, 2011 at 11:08
  • 1
    $\begingroup$ Sounds like you need exactly a probability model with no need for weights. $\endgroup$ Commented Aug 17, 2011 at 7:24
  • 1
    $\begingroup$ Right; plug in the cost function and use the predicted probability and you have an optimal decision. $\endgroup$ Commented Nov 19, 2015 at 13:52
  • 1
    $\begingroup$ With a well-calibrated probability model there are no "errors", there's just randomness that cannot be predicted. Optimal decisions are a function of the predicted probability and the cost function for making various decisions to act. $\endgroup$ Commented Nov 20, 2015 at 13:22
  • 1
    $\begingroup$ Stick to maximum likelihood estimation and you'll be OK. If you want to optimize decision making and not just make predictions, you need a utility function to feed the predicted probabilities into. $\endgroup$ Commented Aug 12, 2017 at 10:56

2 Answers 2

3
$\begingroup$

glm holds a parameter weights exactly for this purpose. You provide it with a vector of numbers on any scale, that holds the same number of weights as you have observations.

I only now realize that you may not be talking R. If not, you might want to.

$\endgroup$
6
  • $\begingroup$ I am very familiar with R, however I'd like to understand the math behind the likelihood function. I might code this in C++ or some other language. (Just trusting the "blackbox" of the glm function isn't always the best solution) $\endgroup$
    – Noah
    Commented Aug 16, 2011 at 8:04
  • $\begingroup$ Ah. Good on you. Well, as far as I know, the weights are simply used to multiply the per-observation loglikelihood with. So if you've written an unweighted version, adding the weights should be a doddle. Note also that you can always look at the source code for glm to (probably) find a C implementation. $\endgroup$
    – Nick Sabbe
    Commented Aug 16, 2011 at 8:15
  • 2
    $\begingroup$ @Nick, I too was under the misconception that this was the function of the weights argument in glm - it is not. It is actually used for when the binomial outcomes are inhomogeneous in the sense that they are based on different numbers of trials. For example, if the first observation was Binomial($3,.5$) and the second was Binomial($7,.5$), their weights would be $3,7$. Again, the weights argument in glm() are NOT sampling weights. To do this in R you will need to expand the data set according to the weights and fit the model to the expanded data set (the SEs may be wrong in this case though). $\endgroup$
    – Macro
    Commented Aug 16, 2011 at 14:12
  • 3
    $\begingroup$ Here is a discussion of the 'weights' argument on a message board: r.789695.n4.nabble.com/Weights-in-binomial-glm-td1991249.html $\endgroup$
    – Macro
    Commented Aug 16, 2011 at 14:18
  • $\begingroup$ @Macro: thx! Very neat. One of the things that could have hit me in the teeth if I'd used it before your comment :-) $\endgroup$
    – Nick Sabbe
    Commented Aug 16, 2011 at 14:28
1
$\begingroup$

If you have access to SAS, this is very easily accomplished using PROC GENMOD. As long as each observation has a weight variable, the use of the weight statement will allow you do perform the kind of analysis you're looking for. I've mostly used it using Inverse-Probability-of-Treatment weights, but I see no reason why you couldn't assign weights to your data to emphasize certain types of cases, so long as you make sure your N remains constant. You'll also want to make sure to include some sort of ID variable, because technically the upweighted cases are repeated observations. Example code, with an observation ID of 'id' and a weight variable of 'wt':

proc genmod data=work.dataset descending;
    class id;
    model exposure = outcome covariate / dist=bin link=logit;
    weight wt;
    repeated subject=id/type=ind;
run;
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.