7
$\begingroup$

The short version:

I can fit a model using Weighted Least Squares, given a diagonal matrix of weights $W$, by solving $(X^TWX)\hat{\beta}=X^TWy$ for $\hat{\beta}$.

Is there a GLM analogue? if so, what is it?

There seems to be a GLM analogue, e.g. with the weights argument in R's glm function. How is R using these weights?

The long version:

the situation

As a follow-up to my IPTW question, I just want to double check that I understand how to fit a parametric model using inverse probability(-of-treatment) weights (IPTW). The idea with IPTW is to simulate a dataset in which the relationship between my independent variables $(a^1,a^2,a^3)$ and dependent variable $y$ is unconfounded and therefore causal. For argument's sake let's say I already estimated an IPT weight $\hat{w}_i$ for each observation. These weights are hypothetical probability weights from the simulated dataset.

the question

I now want to fit a GLM. I'd just use WLS, but I'm working with a binary outcome and an outcome truncated at zero. So I have a linear model $\eta_i=a^T\beta$, a link $\mu_i=g(\eta_i)$, and a variance $V(y_i)$ derived from my likelihood for $y$. Then the likelihood equations are $$ \sum_{i=1}^N \frac{y_i-\mu_i}{V(y_i)}\frac{\partial\mu_i}{\partial\beta_j}=\sum_{i=1}^N \frac{y_i-\mu_i}{V(y_i)}\left(\frac{\partial\mu_i}{\partial\eta_i}x_{ij}\right)=0,~\forall j $$ as per Categorical Data Analysis, Agresti, 2013, section 4.4.5.

So all I have to do is multiply $var(\mu_i)$ by the weight $\hat{w}_i$, right? The same way I might if I wanted to incorporate an overdispersion parameter? If so, is this because the variance of, say, 5 independent observations is 5 times the variance of one independent observation?

Follow-up idea: since the likelihood is the product of the likelihood for each observation, is there some weighting procedure I can use to just weight the likelihoods?

Improve this question
$\endgroup$
9
  • $\begingroup$ This looks relevant, I have not tried it out though: R package for inverse probability weighing $\endgroup$
    – bdeonovic
    Commented Jul 7, 2014 at 13:12
  • $\begingroup$ Saw that. Might end up using it but they don't go through the math of actually estimating the parameters, which I would still need. $\endgroup$
    – shadowtalker
    Commented Jul 7, 2014 at 13:14
  • $\begingroup$ Did you also follow up on: "The use of IPW to fit an MSM was described in detail, e.g., in Robins, Hern´an, and Brumback (2000), Hern´an and Robins (2006) and Cole and Hern´an (2008)."? $\endgroup$
    – bdeonovic
    Commented Jul 7, 2014 at 14:14
  • $\begingroup$ Yep. Every single paper I've read on the subject either says nothing, or says to use PROC GENMOD in SAS, or the equivalent Stata command. I don't like fitting models that I don't understand, plus I already have everything in R. $\endgroup$
    – shadowtalker
    Commented Jul 7, 2014 at 15:55
  • $\begingroup$ Some promise here eml.berkeley.edu/symposia/nsf99/papers/robins.pdf on pages 13-17, but the paper is near-unreadably dense and a bit over my head. $\endgroup$
    – shadowtalker
    Commented Jul 7, 2014 at 15:56

1 Answer 1

Reset to default
7
$\begingroup$

Fit an MLE by maximizing $$ l(\mathbf{\theta};\mathbf{y})=\sum_{i=1}^Nl{\left(\theta;y_i\right)} $$

where $l$ is the log-likelihood. Fitting an MLE with inverse-probability (i.e. frequency) weights entails modifying the log-likelihood to:

$$ l(\mathbf{\theta};\mathbf{y})=\sum_{i=1}^Nw_i~l{\left(\theta;y_i\right)}. $$

In the GLM case, this reduces to solving $$ \sum_{i=1}^N w_i\frac{y_i-\mu_i}{V(y_i)}\left(\frac{\partial\mu_i}{\partial\eta_i}x_{ij}\right)=0,~\forall j $$

Source: page 119 of http://www.ssicentral.com/lisrel/techdocs/sglim.pdf, linked at http://www.ssicentral.com/lisrel/resources.html#t. It's the "Generalized Linear Modeling" chapter (chapter 3) of the LISREL "technical documents."

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ That this is the right formulation for frequency weights is clear when you consider doubling or tripling, etc., individual data values: the factor $\exp(l(\theta, y_i))$ (there's no subscript on the $\theta$) in the likelihood becomes $w_i$ multiplicative factors amounting to $\prod_{j=1}^{w_i}\exp(l(\theta, y_i))=\exp(w_il(\theta,y_i))$ whose logarithm is $w_il(\theta,y_i)$ and there you have it. Different developments (based on conditional probability calculations) are needed for probability weights. $\endgroup$
    – whuber ♦
    Commented Jul 8, 2014 at 20:27
  • $\begingroup$ Makes sense. I just wasn't sure that the analogy held in the case when "frequencies" didn't have to be integers. And, yes, the probabilities I'm inverting are coming from estimated conditional probabilities. $\endgroup$
    – shadowtalker
    Commented Jul 8, 2014 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.