Burnham and Anderson use a Monte Carlo simulation to evaluate selecting between the normal (N) and the log-normal (LN) models. I extend this simulation to select between the normal and the log-normal when the data comes in two batches (drawn from the same data generating process — either normal or log-normal). Empirically and under these idealized conditions, the simulation suggests that multiplying likelihood ratios across datasets is a reasonable approach.
Part I. Model selection with one dataset
I start by reproducing Table 6.14 to make sure I've set up the simulation correctly and to introduce the two evaluation measures.
(a) $\pi$ is the fraction of repeated experiments when the normal model, N, is selected over the log-normal model, LN, because it has higher likelihood:
$$
\begin{aligned}
\operatorname{I}\left\{\frac{L_{\text{N}}}{\operatorname{L}_{\text{LN}}} > 1\right\} = \operatorname{I}\left\{\ell_{\text{N}} > \ell_{\text{LN}}\right\}
\end{aligned}
$$
where $L$ is the likelihood and $\ell$ is the log-likelihood.
(b) $\operatorname{E}(w)$ is the average Akaike weight for the normal model.
This concept was new to me. The Akaike weight is defined in terms of the difference in AIC with the best candidate model, ie, the model with the smallest AIC. Here we are comparing two models with the same complexity (number of parameters), so the model with the smaller AIC has the higher likelihood.
$$
\begin{aligned}
w_{\text{N}} = \frac{\exp\left\{\ell_{\text{N}} - \ell_{\text{best}}\right\}}{\exp\left\{\ell_\text{N} - \ell_{\text{best}}\right\} + \exp\left\{\ell_{\text{LN}} - \ell_{\text{best}}\right\}}
= \frac{1}{1 + \exp\left\{\ell_{\text{LN}} - \ell_{\text{N}}\right\}}
\end{aligned}
$$
While $\pi$ indicates whether we select the correct model or not, the Akaike weights scale the difference $\ell_{\text{LN}} - \ell_{\text{N}}$ to a measure between 0 and 1. While it's wrong to interpret $w_{\text{N}}$ as the "probability that the normal model is correct", the weights indicate whether it is easy or difficult to discriminate between the models. Here is example: Say the data is normal and $\ell_{\text{N}} = -11$, $\ell_{\text{LN}} = -11.2$. Since the normal likelihood is higher, we choose the normal model, correctly; its Akaike weight is 0.55. (The Akaike weight of the log-normal model is 045.) Now if we had ~10x more data and the log-likelihoods are $\ell_{\text{N}} = -110$, $\ell_{\text{LN}} = -112$, we still select the correct model by comparing the likelhoods; but now $w_{\text{N}} = 0.88$. We take the same decision but with more "certainty".
And here is my version of Table 6.14; the results are close to those in the book. It's harder to discriminate between the two models when there is less data (we can tackle this by getting more data) and when the mean is larger (the distributions are very similar, so the model choice makes little difference).
n |
data |
μ=5: π |
μ=5: E(w) |
μ=10: π |
μ=10: E(w) |
μ=25: π |
μ=25: E(w) |
μ=50: π |
μ=50: E(w) |
10 |
normal |
58 |
54 |
54 |
51 |
51 |
50 |
51 |
50 |
50 |
normal |
78 |
71 |
65 |
57 |
56 |
51 |
53 |
50 |
100 |
normal |
87 |
83 |
72 |
63 |
59 |
53 |
54 |
51 |
500 |
normal |
100 |
100 |
91 |
87 |
71 |
62 |
61 |
53 |
10 |
log-normal |
38 |
46 |
44 |
49 |
48 |
50 |
49 |
50 |
50 |
log-normal |
21 |
29 |
34 |
43 |
43 |
49 |
47 |
50 |
100 |
log-normal |
12 |
18 |
28 |
37 |
41 |
47 |
45 |
49 |
500 |
log-normal |
0 |
1 |
9 |
13 |
29 |
38 |
39 |
47 |
Part II. Model selection with two datasets
Now that I can run the simulation for one dataset of size $n$, it's easy to extend it to two datasets of size $n_1$ and $n_2$. For comparison with the previous results I choose the sample sizes so that $n_1 + n_2 = n$, for $n = \{25, 50, 100, 500\}$ split into ratios $9:1$, $4:1$, $2:1$ and $1:1$. [I omit $n = 10$ as it's too small to split and compute the MLEs.]
Here is how I compute the pooled measures:
(a) Exactly as suggested, $\pi$ is the fraction of repeated experiments when the normal model, N, is selected over the log-normal model, LN:
$$
\begin{aligned}
\operatorname{I}\left\{\frac{L_{\text{N}}^{(1)}}{\operatorname{L}_{\text{LN}}^{(1)}}\frac{L_{\text{N}}^{(2)}}{\operatorname{L}_{\text{LN}}^{(2)}} > 1\right\} = \operatorname{I}\left\{\ell_{\text{N}}^{(1)} + \ell_{\text{N}}^{(2)} > \ell_{\text{LN}}^{(1)} + \ell_{\text{LN}}^{(2)}\right\}
\end{aligned}
$$
where the superscripts indicate datasets $1$ and $2$.
(b) The average Akaike weight for the normal model across the two datasets is:
$$
\begin{aligned}
w_{\text{N}} = \frac{\sum_{i}\exp\left\{\ell_{\text{N}}^{(i)} - \ell_{\text{best}}^{(i)}\right\}}{\sum_{i}\exp\left\{\ell_\text{N}^{(i)} - \ell_{\text{best}}^{(i)}\right\} + \exp\left\{\ell_{\text{LN}}^{(i)} - \ell_{\text{best}}^{(i)}\right\}}
\end{aligned}
$$
The log-likelihood is normalized separately for each dataset. So we can't compute the pooled Akaike weight from the likelihood ratio alone; we need to have the likelihood of each model separately.
I summarize the results in a small-multiples plot. There is a separate panel for each combination of sample size $n=\{25, 50, 100, 500\}$ and population mean $\mu=\{5, 10, 25, 50\}$. The dotted horizontal lines indicate the model selection performance (in terms of $\pi$ in blue and $\operatorname{E}(w)$ in red) when we analyze one dataset of size $n$. The solid lines indicate the "pooled" performance with $n$ samples split across two datasets according to the ratio given on the x-axis.
Figure 1: Selecting between the normal model (correct) and log-normal model (wrong) by pooling log-likelihood ratios between two datasets.

As Burnham and Anderson explain, $\pi$ reflects sampling variation (how often we select the correct model) and $\operatorname{E}(w)$ reflects inferential uncertainty (how easy it is to discriminate between the two models).
In this simulation at least, we don't lose much accuracy to select the correct model unless $\mu$ $=$ $5$ or the (total) sample size is $n$ $\leq$ $50$: the solid blue line is only slightly below the dotted blue line. But there is more uncertainty (big gap between the solid and dotted red lines) and it's worse to have the samples unevenly split between the two datasets. So we lose some information by splitting the data and the estimation into two; this is not surprising. In this simulation there is no order in how the datasets are generated. But in practice the datasets could be collected & analyzed sequentially. There may be little gain to collect a small followup sample and study it separately. It would be better to put the datasets together and analyze all the data we have, esp. if as stated in the question, the datasets are generated under similar conditions.
Figure 2: Selecting between the normal model (wrong) and log-normal model (correct) by pooling log-likelihood ratios between two datasets.

The R code to reproduce the simulation (mostly in base R) and the figure (with ggplot2).
R <- 100000
# Find the MLEs of the normal mean and variance; Compute the log-likelihood.
mle.norm <- function(x) {
xbar <- mean(x)
s2 <- mean((x - xbar)^2)
sum(dnorm(x, xbar, sqrt(s2), log = TRUE))
}
# Find the MLEs of the log-normal mean and variance; Compute the log-likelihood.
mle.lnorm <- function(x) {
lxbar <- mean(log(x))
ls2 <- mean((log(x) - lxbar)^2)
sum(dlnorm(x, lxbar, sqrt(ls2), log = TRUE))
}
# Generate normal data; Maximize the normal and log-normal log-likelihoods.
fit.norm <- function(n, mu) {
sigma <- 1
x <- rnorm(n, mu, 1)
c(ll.a = mle.norm(x), ll.b = mle.lnorm(x))
}
# Generate log-normal data; Maximize the normal and log-normal log-likelihoods.
fit.lnorm <- function(n, mu) {
sigma <- 1
phi <- sqrt(mu^2 + sigma^2)
x <- rlnorm(n, log(mu^2 / phi), sqrt(log(phi^2 / mu^2)))
c(ll.a = mle.norm(x), ll.b = mle.lnorm(x))
}
# ---
# Do model selection using log-likelihood ratio & Akaike weights.
# NB. Since the two candidate models -- normal and log-normal -- have the same
# number of parameters, AIC(M1) - AIC(M2) = LL(M1) - LL(M2) = log{L(M1)/L(M2)}
select.model <- function(lls) {
lls %>%
drop_na() %>%
mutate(
better = ll.a > ll.b,
Δ.a = ll.a - pmax(ll.a, ll.b),
Δ.b = ll.b - pmax(ll.a, ll.b),
weight = exp(Δ.a) / (exp(Δ.a) + exp(Δ.b))
) %>%
summarise(
π = mean(better),
`E(w)` = mean(weight)
)
}
select.model.2batches <- function(lls) {
lls %>%
drop_na() %>%
mutate(
better = (ll.a1 + ll.a2) > (ll.b1 + ll.b2),
Δ.a1 = ll.a1 - pmax(ll.a1, ll.b1),
Δ.b1 = ll.b1 - pmax(ll.a1, ll.b1),
Δ.a2 = ll.a2 - pmax(ll.a2, ll.b2),
Δ.b2 = ll.b2 - pmax(ll.a2, ll.b2),
weight = (exp(Δ.a1) + exp(Δ.a2)) / (exp(Δ.a1) + exp(Δ.b1) + exp(Δ.a2) + exp(Δ.b2))
) %>%
summarise(
π = mean(better),
`E(w)` = mean(weight)
)
}
# ---
# Boilerplate code to run the simulations
simulate.norm.data <- function(n, mu, reps = R) {
seq(reps) %>% map_dfr(~ fit.norm(n, mu))
}
simulate.lnorm.data <- function(n, mu, reps = R) {
seq(reps) %>% map_dfr(~ fit.lnorm(n, mu))
}
simulate.norm.data.2batches <- function(n, mu, p, reps = R) {
n1 <- round(n * p)
n2 <- n - n1
data1 <- simulate.norm.data(n1, mu) %>% rename_with(~ str_c(., "1"))
data2 <- simulate.norm.data(n2, mu) %>% rename_with(~ str_c(., "2"))
bind_cols(data1, data2)
}
simulate.lnorm.data.2batches <- function(n, mu, p, reps = R) {
n1 <- round(n * p)
n2 <- n - n1
data1 <- simulate.lnorm.data(n1, mu) %>% rename_with(~ str_c(., "1"))
data2 <- simulate.lnorm.data(n2, mu) %>% rename_with(~ str_c(., "2"))
bind_cols(data1, data2)
}
run.simulations.save.results <- function(seed = 1234) {
set.seed(seed)
# Table 6.14A
grid <- expand_grid(
n = c(10, 25, 50, 100, 500),
μ = c(5, 10, 25, 50)
)
sim.norm <- grid %>%
mutate(
data = pmap(list(n, μ), simulate.norm.data),
table = map(data, select.model)
)
sim.lnorm <- grid %>%
mutate(
data = pmap(list(n, μ), simulate.lnorm.data),
table = map(data, select.model)
)
bind_rows(
norm = sim.norm,
lnorm = sim.lnorm,
.id = "data"
) %>%
unnest(
table
) %>%
write_csv(
"600592-Table-6.14A.csv"
)
# Table 6.14B
grid <- expand_grid(
n = c(25, 50, 100, 500),
μ = c(5, 10, 25, 50),
p = c(1 / 10, 1 / 5, 1 / 3, 1 / 2)
)
sim.norm <- grid %>%
mutate(
data = pmap(list(n, μ, p), simulate.norm.data.2batches),
table = map(data, select.model.2batches)
)
sim.lnorm <- grid %>%
mutate(
data = pmap(list(n, μ, p), simulate.lnorm.data.2batches),
table = map(data, select.model.2batches)
)
bind_rows(
norm = sim.norm,
lnorm = sim.lnorm,
.id = "data"
) %>%
unnest(
table
) %>%
write_csv(
"600592-Table-6.14B.csv"
)
}
# 6.7.2 A Normal Versus Log-Normal Example
library("tidyverse")
plot_selection <- function(data) {
table.b %>%
filter(data == {{ data }}) %>%
ggplot(
aes(
p, value,
group = name,
color = name
)
) +
geom_line() +
geom_hline(
aes(
yintercept = value,
color = name
),
linetype = 2,
data = table.a %>% filter(data == {{ data }})
) +
facet_wrap(
~ n + μ,
scales = "free_y",
labeller = "label_both"
) +
scale_x_continuous(
breaks = c(1 / 10, 1 / 5, 1 / 3, 1 / 2),
labels = c("1:9", "1:4", "1:2", "1:1")
) +
theme(
axis.title = element_blank(),
legend.title = element_blank(),
panel.spacing.y = unit(0, "line")
)
}
run.simulations.save.results()
table.a <- read_csv("600592-Table-6.14A.csv")
table.b <- read_csv("600592-Table-6.14B.csv")
table.a %>%
pivot_wider(
names_from = μ,
names_glue = "{μ},{.value}",
values_from = c(π, `E(w)`),
values_fn = ~ round(100 * .)
) %>%
select(
n, data,
starts_with("5,"),
starts_with("10,"),
starts_with("25,"),
starts_with("50,")
) %>%
knitr::kable()
table.a <- table.a %>%
filter(n >= 25) %>%
pivot_longer(c(π, `E(w)`))
table.b <- table.b %>%
filter(n >= 25) %>%
pivot_longer(c(π, `E(w)`))
plot_selection("norm")
plot_selection("lnorm")