# Model selection for this model with one observation

I would like to perform model selection given a range of $$k$$ models $$\mathcal{M}_1, \mathcal{M}_2, ..., \mathcal{M}_k$$, each with some prior probability $$f(\mathcal{M}_1), \dots, f(\mathcal{M}_k).$$

I have a probability distribution for the random variable $$X$$ given each $$\mathcal{M}_i$$ and a vector of unknown parameters $$\boldsymbol{\theta} \in \mathbb{R}^{m}$$, where $$m > 1$$. This is to say that I know and can compute the probability mass function $$f(x|\boldsymbol{\theta},\mathcal{M}_i)$$. I do not have this in closed form but I can evaluate it.

The parameter vector $$\boldsymbol{\theta}$$ is independent of $$\mathcal{M}_i$$, that is to say $$f( \boldsymbol{\theta}|\mathcal{M}_i) = f(\boldsymbol{\theta})$$ for each model.

My data is in the form $$\mathcal{D} = x \in \mathbb{N}$$. Unfortunately, $$\boldsymbol{\theta}$$ is unidentifiable (via any MLE/MCMC method) from my single data point $$x$$.

My question is, how (if at possible) can I perform model selection (or any kind of hypothesis test) given my single data point $$x$$?

My idea was to use Monte Carlo Integration to find: $$f(\mathcal{D}|\mathcal{M}_k) = \int_\Theta f(\mathcal{D}|\boldsymbol{\theta},\mathcal{M}_k) f(\boldsymbol{\theta}) d \boldsymbol{\theta} \approx \frac{1}{N} \sum_{i=1}^{N} f(\mathcal{D}|\boldsymbol{\theta}^{(i)},\mathcal{M}_k),$$ where $$\boldsymbol{\theta}^{(i)}$$ for $$i = 1, ... , N$$ are iid samples from $$f(\boldsymbol{\theta})$$. Then I can compute $$f(\mathcal{M}_i|\mathcal{D}) = \frac{f(\mathcal{D}|\mathcal{M}_i) f(\mathcal{M}_i) }{\sum_j f(\mathcal{D}|\mathcal{M}_j) f(\mathcal{M}_j)},$$ for each model $$i =1, \dots, k$$ and then either pick the model which maximises this posterior likelihood or use this to compute Bayes Factors.

Is this a standard technique used? Or am I missing something? How would this relate to hypothesis testing? Thanks!

• The (proper) prior removes the identifiability issue and makes the marginal likelihood well defined. From there it is indeed a correct Bayesian approach to the problem. With a single integer observation the support for one model versus another will proceed mostly from the corresponding priors. – Xi'an Mar 30 '19 at 10:58