I'm trying to estimate a regression where:
- I can only see the sex of a subset of the population
- I do know the total population size
- I'd like to know how many events are driven my men vs women, using a regression model
In some cases, I only have a very small sample where I know the sex of the population, so I'd like to use a multilevel model to fill those periods in using a prior based on all periods.
Thus, I estimate the proportion of women on each day, then use that proportion to infer the total male/female population size, then use those two sizes to build my regression.
My problem is that my model is finding degenerate solutions by setting the proportion of females to exactly match the number of events.
I can solve this by doing two separate estimations, one of the proportion of females, using a multilevel model, then use the average estimate from that model to build my regression, but this will not properly account for uncertainty in the estimate of the proportion.
Is there a way to "block" effects in these models, much as I might "block" a gradient in gradient descent to avoid the second part of the model from affecting the first?
For reference, I'm doing this in PyMC3, but I'd be happy to move to Stan or another tool if warranted.
Here's the models's pseudocode:
prob_female ~ Uniform(0, 1) scale_female ~ HalfNormal(5) # I'm using PyMC3, which has a location/scale parameterization # of the beta distribution female_rate ~ Beta(mu=prob_female, sd=scale_female, shape=n_days) observed_female ~ Binomial(p=female_rate, n=day_visitors, observed=female_visits) # This is where the "block" should be, dividing # these two sections inferred_female = observed_female * day_visitors inferred_male = (1-observed_female) * day_visitors female_event_rate ~ Normal(0, 5) male_event_rate ~ Normal(0, 5) event_sd ~ HalfNormal(5) event_rate = inferred_female * female_event_rate + inferred_male * male_event_rate observed_events ~ Normal(event_rate, event_sd, observed=n_events)