The Ambiguity in Schwarz Information Criterion Definition

Question

Suppose there are 100 countries, $i = 1, 2, ..., 100$. Let $b_i$ be the median birth weight of all new born boys in country #i in 2019. Let $g_i$ be the median birth weight of all new born girls in country #i in 2019. We want to model data set {$b_1, g_1, b_2, g_2, ..., b_{100}, g_{100}$} with two equivalent models.

Model #1. Ordinary Least Square

Under this model, we have 200 observations whose dependent variable values are those above and whose independent variable values are 0, 1 dummy variable $d_j$ to indicate if it is a boy or not:

$z_j = x + y * d_j + e_j$ where $e_j$ is IID normal, for $j = 1, 2, ..., 200$.

Model #2. Constrained 0th-Order Vector Auto Regression

Under this model, we have 100 observations, each of which is a 2x1 vector. $Z_j = (x, y)' + E_j$ where $E_j$ is IID normal, with a diag covariance matrix and equal variances, for $j = 1, 2, ..., 100$.

Ambiguity

Notice that these two models are mathematically equivalent, with the same maximum likelihood estimates for $(x, y)$ and the same maximized log-likelihood function values.

But the "numbers of observations" differ. So the Schwarz information criterion values differ. This is troublesome. What is the proper Schwarz information criterion in this situation?

---

Thanks Eric, for your detailed response. Sorry about my late reply.

I was not entirely explicit. I was not trying to use BIC to choose between "model 1" and "model 2", as they are equivalent. I should have phrased them as "approaches" rather than "models".

Formally and using the revised proper terms, I intend to, for example, use BIC to choose between two models (for the SAME data), whereas the model 1 is as I described above ("unrestricted"), while model 2 has an extra constraint $y = 0$ ("restricted"). Now for both models, I can parameterized them through either approach 1 (OLS) or approach 2 (VAR). Under these two different approaches, it is possible that BIC would favor either the restricted or unrestricted model differently.

This bothers me, as these two approaches are equally valid.

---

Eric, I agree with you, when $d_j$ is stochastic, that the mixed normal would fall out of the exponential family and thus invalidates the key assumption of Schwarz (1978).

But what about the case, when $d_j$ is exogeneous/deterministic, subject to the technicality condition (just as in the classical OLS asymptotic setting), $\frac{\sum_1^N d_j}{N}$ converges to some constant (e.g. 0.5)? In this case, our data set is $(z_j, d_j)_{j=1,2,...}$ with $d_j$ being degenerate, and the likelihood function would be exactly the same as that for VAR (and thus remains in the exponential family).

The key feature of the exponential family is that data and parameter "mix" in only one place. Maybe there is an additional implicit "mixing" between $d_j$ and parameters, despite the identical-ness of likelihood functions on the surface? I need to think more about this. Love to hear your thoughts.

Answer 1

Edit

Thanks for the clarifying point. In this case, the vector model (Approach #2) is the correct one, and the dummy variable model (Approach #1) is wrong.

This is because Approach #1 doesn't fit the assumptions of the Schwarz criterion, because the distribution of the observations $z$ does not follow a distribution from the exponential family. Because of the dummy variable $d$, the distribution of $z$ would actually be a mixture of two normal distributions, one for boys with mean $x + y$ and the other for girls with mean $x$. Mixture models are not generally members of the exponential family, see the last paragraph of the "Examples" section of the Wikipedia article for the exponential family.

Approach #2, on the other hand, has $z$ distributed according to the normal distribution $$ \mathcal{N}\left( \left[ \begin{matrix} x + y \\ x \end{matrix} \right] , \left[ \begin{matrix} \sigma^2 & 0 \\ 0 & \sigma^2 \end{matrix} \right] \right)$$ which is a member of the exponential family. So Approach #2 is the correct one, and the correct number of observations is $n = 100$.

Original Answer

The data is not allowed to differ when using the Schwarz criterion to compare two models. To quote from Schwarz's original paper

In a general parameter space, there is no intrinsic linear structure. We therefore assume that observations come from a Koopman-Darmois [Exponential] family, i.e., relative to some fixed measure on the sample space they possess a density of the form $$ f(x, \theta) = \exp(\theta \cdot y(x) - b(\theta)). $$ where $\theta$ ranges over the natural parameter space $\Theta$, a convex subset of the $K$-dimensional Euclidean space, and $y$ is the sufficient $K$-dimensional statistic. The competing models are given by sets of the form $m_j \cap \Theta$ where $m_j$ is a $k_j$-dimensional linear manifold embedded in $\mathbb{R}^K$ for each $j$.

The two models in your question don't satisfy the assumptions for this setup, because you are changing the data $x$ between the two models (notice that $x$ does not have a $j$ subscript in the above). If you want to use the Schwarz information criterion (BIC) then you have to compare two models that satisfy these assumptions, which includes using the same data $x$. In particular, $$ \{ b(1), g(1), b(2), g(2), ..., b(100), g(100) \} \ne \left\{ \left[ \begin{matrix} b(1) \\ g(1) \end{matrix} \right] , \left[ \begin{matrix} b(2) \\ g(2) \end{matrix} \right], ..., \left[ \begin{matrix} b(100) \\ g(100) \end{matrix} \right] \right\}. $$