Mixed Effects, Doctors & Operations: predicting on new data containing previously unobserved levels, and updating our confidence accordingly Here's a quick sketch of a hypothetical situation. There are Doctors $\{1, \ldots, J\}$  who perform different types of operations $\{1, \ldots, K\}$.  Our response variable is whether the operation was a success ($Y_{i,j,k} = 1$ implying that for the $i$th patient for doctor $j$ for which he performed operation $k$ was a success).
Doctors have some inherent level of skill $\alpha_j$, and operations have some inherent level of difficulty $\beta_k$. We might think that there are some fixed known number of operations, but that Doctor skill is normally distributed, drawn from a population of doctors. So we assume that Doctor skill is distributed as $\alpha_j \sim N(0, \sigma_\alpha^2)$.
A crude model might look something like the following, where the Logit of the probability of success is a sum of some base intercept, the skill of the doctor, and the difficulty of the operation. 
$$
\text{Logit}(\text{Pr}(Y_{i,j,k} = 1)) = \mu_0 + \alpha_j + \beta_k.
$$
This naive model seems to be a straightforward mixed effects model. Doctors are levels we observe from some larger population, and their skills are random intercepts. The Operations can be categorical fixed effects (determining difficulty). In reality we would certainly expect there to be an interaction between Doctor and Operation (they specialize!), but I'll ignore that for now as I'm just trying to understand a theoretical example.
Here's the crux of the issue. I want an approximate model (this need not be perfectly rigorous) which can look at a new batch of data and guess "About how many failed operations do we expect?". This is some function of the skill of the doctors involved, how many operations they do, and what those operations are. However, in practice, my new batch of data will contain a mix of doctors which I know a great deal about, as well as new recruits fresh out of med school! (Who lack previous data to indicate their skill).
Does a mixed effects model still work in this case? Basically, I want a model which defaults to some sort of baseline assumption of skill when we haven't observed this doctor in practice, and then updates it as we observe more (until we are very confident if we have a large sample for them). This is clearly a Bayesian framework: we have some prior belief about their skill, and we update it to reflect new information. 
As a concrete example, our new data contains operations from Doctors A, B, and C. Doctor A is a veteran who has performed 5,000 operations, and we can confidently estimate that they are much more skilled than average as they rarely fail. Doctor B just moved from a different hospital, so we assume they are exactly as skilled as the average doctor. Doctor C has performed 100 operations with a higher failure rate than usual, so we think they are less skilled but recognize that it's a small sample so we hedge our bets a bit. Again, this is an obviously bayesian set-up, but I'm not sure if it can be accomplished using a mixed effects model, or how to actually build a reasonable bayesian model to reflect this.
So, in total,


*

*Can a mixed effects model work here? (That is, update its confidence on doctor skill based on how much past data we've seen for this specific random intercept).

*Should I instead move to a more directly Bayesian framework? (I'm sure the mixed effects model has a Bayesian interpretation, but again I'm not sure if it specifically does what I need here).

*Do you have any ideas for models which might do what I need here? This is meant to be a rough benchmark, not an elite predictive model, so simple and intuitive is fine even if it doesn't exactly fit the scenario (the Doctors and Operations is just an example anyways).


Thank you all for any advice you can provide! 
 A: What you describe is the concept of dynamic predictions from mixed models. 
Initially, when you have no information for a doctor you only use the fixed effects in the prediction, i.e., you put his/her ability level equal to the average ($\alpha_j = 0$). 
But, as extra information is recorded you can update your predictions by calculating the random effect of the doctor. You get this from the posterior/conditional distribution of the random effects $\alpha_j$ given the observed data $y_j$.
You can find more information on these predictions in the Dynamic Predictions vignette of the GLMMadaptive package.
A: I think a Bayesian approach might be beneficial. When predicting for an unobserved category in a Bayesian mixed model, you generate posterior predictions by sampling the $\alpha_j$ from the fitted $N(0, \sigma^2_\alpha)$ (aside from sampling the fixed effects from the fitted posterior). This way, you will see high uncertainty for the new doctor - which is IMHO a good thing. The amount of uncertainty will be related to the fitted $\sigma^2_\alpha$, so when most doctors are similar, you will get lower uncertainty. After you have more data, you would refit the model and the uncertainty for the (now observed) doctor will gradually shrink.
