Multinomial/Categorical models with a variable number of outcomes

Question

Is it possible to implement a multinomial/categorical model where the number of categories itself is variable? For example, say I have two surveys with the following questions/responses:

• What ice cream flavor do you prefer: Chocolate/Vanilla/Strawberry; 1,000 / 900 / 100
• What ice cream flavor do you prefer: Chocolate/Vanilla/Strawberry/Neopolitan; 800 / 700 / 100 / 400

It feels like we should be able to use the information about the first question (i.e., chocolate/vanilla are roughly even, strawberry isn't preferred) in a big 'ole model with the second question, but I haven't seen this done anywhere.

If feasible, I'm also wondering which is a better method for modeling --- using two separate response scales or using one but conditionally fixing one of the responses to 0.

For example, here the probability vector's length depends on the number of possible categories (so the lengths of $p_{k1}$ and $p_{k2}$ are 3 and 4, respectively:

$$ \begin{align*} R_i | \text{question 1} & \sim \text{Multinomial}(n, p_{k1}) \\ R_i | \text{question 2} & \sim \text{Multinomail}(n, p_{k2}) \end{align*} $$

Or, alternatively:

$$ \begin{align*} R_i & \sim \text{Multinomial}(n, p_k) \\ \text{logit}(p_1) & = \text{some model for p1} \\ \text{logit}(p_2) & = \text{some model for p2} \\ \text{logit}(p_3) & = \text{some model for p3} \\ \text{logit}(p_4) | \text{question 1} & = 0 \\ \text{logit}(p_4) | \text{question 2} & = \text{some model for p4} \end{align*} $$

Answer 1

Purely mathematically (ignoring all the issues with asking questions in different ways such as tendencies to consider all options when they are offered), you could of course assume that in the first case you got an answer to Chocolate/Vanilla/Strawberry assuming only those are on offer vs. in the second case Chocolate/Vanilla/Strawberry/Neopolitan. You could then assume that, if you have probabilities $p_1$, $p_2$ and $p_3$ of picking each flavor in the first case (multinomial likelihood, as you say) that in the second case you get probabilities $p_1' = p_1 \times (1-p_{14})$, $p_2' = p_2\times (1-p_{24})$, $p_3' = p_3\times (1-p_{34})$ and $p_4' = p_1 p_{14} + p_2 p_{24} + p_3 p_{34}$, again with a multinomial likelihood.

I.e. you would assume that with probability $p_{14}$ someone that would have preferred 1 in the first experiment this person would instead pick 4 in the second experiment. Presumably, you'd want to allow $p_{14}$ to $p_{34}$ to be different for the different flavors, in case liking one flavor correlates stronger with liking the 4th flavor.

Of course, we can also allow for differences in the experiments (by e.g. letting the probabilities vary between experiments with e.g. random effects) or other things to account for differences.

This all gets a bit messy to write down when there's many experiments with many changing categories (e.g. some experiments both add and remove options), but in principle you could follow the same idea.

If I had to fit such a model in practice, it would seem like Stan (e.g. via rstan or cmdstanr in R or any of the other interfaces) would be a pretty convenient option. It would also let you provide sensible prior distributions (e.g. Dirichlet(0.5, 0.5, 0.5) for the first experiment, and perhaps Beta(1/4, 3/4) or something like that for $p_{i4}$ to indicate a very slight prior expectation to stay with the old choice (as long as the numbers are huge like in your example it shouldn't matter too much).