Is there a word or phrase to describe a model that is basically unidentifiable in practice?

Question

So suppose we have a model $f(x|\theta)$ that is theoretically identifiable, so that $\theta_1 \neq \theta_2$ implies $f(x|\theta_1) \neq f(x|\theta_2)$.

However, suppose that data collection is very limited and that, for all samples $X$ taken in practice, for every $\theta$ there exists a $\hat{\theta}$ such that $f(X|\theta) \approx f(X|\hat{\theta})$.

And an example of this would be if $X=(x_1,\dots,x_n)$ is a time series model in equilibrium, i.e., it is basically constant, over the time period considered, such that $X_i \approx x_c$ for all $i=1,2,3,\dots,N.$

If this was the case, and if for example we had $f(x|\theta) \propto \exp \left(-\prod_i \left(\Delta x_i-\alpha x_i^\beta \right)^2 \right)$, then the model would, for practical considerations, be unidentifiable since we would have

$$\log f(x|\theta) \propto - \prod_i \left(\Delta x_i-\alpha x_i^{\beta_1}\right)^2 \approx - \prod_i \left(\Delta x_i-\alpha x_c^{\beta_1}\right)^2 = -\prod_i \left(\Delta x_i-\tilde{\alpha} x_c^{\beta_2} \right)^2\\ \approx -\prod_i \left(\Delta x_i-\tilde{\alpha} x_i^{\beta_2}\right)^2 $$

for any $\beta_1,\beta_2 > 0$, where $\hat{\alpha} = \alpha x_c^{\beta_1-\beta_2}$. If the above was the case, then it would be practically impossible to determine between $(\alpha,\beta_1)$ and $(\hat{\alpha},\beta_2)$, even though the model is theoretically identifiable. I am curious if there is a word or phrase to describe this situation? And whether my concerns make sense.

Answer 1

Surely this is the case for almost every statistical model with a continuous valued parameter $\theta$ and a likelihood that is reasonably smooth. Perhaps this is an issue of computational precision rather than a statistical issue?

For instance, take the likelihood to be that of a Normal with mean $\theta$ and known variance $1$. This is an identifiable model.

Thus, for a sample size of $1$, $$\log f(x \mid \theta) = C -(x - \theta)^2$$ where $C$ is a scaling constant which ensures the likelihood is valid.

Now $$\log f(x \mid \theta + \delta) = C -(x - \theta - \delta)^2$$.

Now if $\delta$ is sufficiently small relative to $\theta$, (say $\theta = 5$, $\delta = 10^{-20}$) a computer could never tell $\log f(x \mid \theta)$ apart from $\log f(x \mid \theta + \delta)$. There is a mathematical distinction, but if a computer can't tell the difference then, to a modern statistician, they are effectively the same.

On a similar theme, it is in practice almost impossible to tell the difference between a sech distribution and the logistic distribution for certain parameter values, despite them having different probability density functions.