2
$\begingroup$

I've got one independent variable and two dependent variables. As a concrete example, consider the independent variable to be browser (categorical, what Web browser a visitor to a Web site uses) and the dependent variables to be visitors (number of visitors) and enrollers (number of visitors who sign up for a newsletter). Note that enrollers depends on visitors (because every enroller is also a visitor).

I care about the rate enrollers/visitors only. How can I determine the effect size of browser on the rate enrollers/visitors?

My specific concern is that I don't know to study enrollers/visitors where power and presumably effect size depend (also) on visitors itself.

$\endgroup$
6
  • $\begingroup$ There are many ways I believe. I would use logistic regression, but seems like poisson / neg bin regression with an offset for visitors could be used. Here are some other methods: statcompute.wordpress.com. Sorry not much time to elaborate, but all are easy with R. $\endgroup$
    – B_Miner
    Commented May 10, 2012 at 20:56
  • $\begingroup$ @msh210 are you still interested in the answer to this question? $\endgroup$ Commented Jun 16, 2012 at 13:12
  • $\begingroup$ @ConjugatePrior yes. $\endgroup$
    – msh210
    Commented Jun 17, 2012 at 15:44
  • $\begingroup$ @msh210 is there something wrong with the one I provided? $\endgroup$ Commented Jun 17, 2012 at 18:10
  • $\begingroup$ @ConjugatePrior I didn't get notification of it when you posted it, so didn't see it until you posted the above comment, and haven't had a chance to check it thoroughly as of yet. $\endgroup$
    – msh210
    Commented Jun 17, 2012 at 18:19

1 Answer 1

4
$\begingroup$

A first cut logistic regression would be

model <- glm(cbind(enrolled, visitors-enrolled) ~ browser, family=binomial)

where browser is a dummy variable, (a factor in R), and enrollers and visitors are counts within some set of time periods. Use predict to get predicted probability of a particular browser user enrolling once they have visited.

This model treats enrolled as a binomially distributed count with trial length visitors with shifts in enrollment probability due to browser choice (only). Since this is probably not quite true you might think of other things to put on the right hand side.

Here's a worked example with two browser types 'IE' and 'FF'

## Assume: IE has .3 chance of enrolling, FF has .6 chance of enrolling,
##         3/4 of browsers are IE,
##         visits in each of 200 time periods are Poisson distributed.

N <- rpois(200, 15) + 1;      ## visitors per time period, guaranteed positive
fake <- data.frame(enrolled=c(rbinom(rep(1,150), N[1:150], rep(.3,150)),
                              rbinom(rep(1,50), N[151:200], rep(.6,50))), 
                   visitors=N, 
                   browser=c(rep("IE", 150), rep("FF", 50)))

## fit model to fake data
model <- glm(cbind(enrolled, visitors-enrolled) ~ browser, family=binomial, data=fake)

## predict rate of enrollment for each browser type (should recover ~0.3 and ~0.6)
predict(model, newdata=data.frame(browser=c('IE', 'FF')), type='response')

The difference between these two probabilities is one possible measure of the effect of switching browsers. Alternatively you could compute it in proportional increase in probability going from IE to FF with a division...

If the rate of enrollment is very small relative to visits you might be better doing

model <- glm(enrolled ~ browser + offset(log(visitors)), family=poisson, data=fake)

or some over-dispersed variant. Here the probabilities are gotten by exponentiating the constant term (for FF) and then the constant term plus the coefficient for IE. That should also get you about 0.6 and 0.3

We can be a bit more statistical about the effect size by generating a probability distribution for it. Here's a sampling-based approach for the Poisson model above. (We can often do the math in closed form, but who cares when we can always sample...)

## get the distribution of differences in enrollment rates for, say 20 visits

## simulate from the sampling distribution of the parameters
library(MASS) 
sim.param <- mvrnorm(n=500, mu=coef(model), Sigma=vcov(model))

## make a function to compute effect size
eff <- function(x){ exp(x[1] + log(20)) - exp(x[1]+x[2] + log(20)) }
## note: the log(20) above is doing the same thing as the offset above.

## apply it 500 times and summarize the results
diffs <- apply(sim.param, 1, eff)
plot(density(diffs))             ## the distribution of effect sizes
quantile(diffs, c(0.025, .975))  ## a 95% interval

The latter should be between 4.7 and 7 for 20 visits. For other definitions of effect size, you'd just change the eff function to compute them and rerun.

$\endgroup$
2
  • $\begingroup$ Thank you! I'm probably missing something, but don't see how I would get effect size from this. $\endgroup$
    – msh210
    Commented Jun 17, 2012 at 18:35
  • $\begingroup$ @msh210 I've put a bit more in about possible definitions and computations of effect size. While you can also look these up in a book, I think it's helpful to sample instead because this method will generalize. Also, the R code should run when pasted in now. $\endgroup$ Commented Jun 18, 2012 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.