2
$\begingroup$

I understand that a parameterization is identified if it's true that

$$ \theta_1 \neq \theta_2 \Rightarrow p(y|\theta_1) \neq p(y|\theta_2) $$

Intuitively, it means that two different parameter values must result in two different probability of the observed data.

However, I don't understand what people means by "weakly identified" parameterization? Is that when different values of $\theta$ leads to close, by not exactly the same, $p(y|\theta$)?

For example, here's an example where Gelman discusses weak identifiability for Item Response Theory model.

$\endgroup$
1
$\begingroup$

The term "weakly identified," as Gelman is using it in your linked post, means that there is a region of parameter space that produces similar likelihood values.

If you think of the likelihood as a surface, then a weakly identified model has a "flat" likelihood around the maximum likelihood solution. This is problematic because it implies a high degree of uncertainty in your parameter estimate. By contrast, if the likelihood were very "peaked" around the maximum likelihood solution, then the uncertainty in the parameter estimate would be small.

You asked if weak identification referred to the case where different parameter values lead to similar likelihoods. This is partly correct. If these "nearly equivalent" parameters are all in one region of space (the "ridge" referred to by Gelman), then we have weak identifiability. If, however, the likelihood function has several distinct regions of near-maximum likelihood, then it is more common to call the model "locally identifiable." For example, when fitting a Gaussian distribution to observations, the standard deviation $\sigma$ is technically only locally identifiable, because both $\hat\sigma$ and $-\hat\sigma$ are maximum likelihood solutions. However, when we impose the restriction that $\sigma>0$, the model becomes identifiable.

$\endgroup$
2
  • $\begingroup$ To go further with the "ridge" imagery, is it correct to say that: "identifiable model" = the likelihood has a sharp peak. "non-identifiable model" = the likelihood has a ridge. "weakly identifiable" = there is a peak (not very sharp) along the ridge? $\endgroup$ – Heisenberg Sep 14 '20 at 21:18
  • $\begingroup$ This is the right idea. I would point out that a non-identifiable model is any model where two parameter settings give the same likelihood. So, for example, the locally identifiable case is a special type of non-identifiability, but the likelihood there is not a ridge. For an example where it is a ridge, consider the (weirdly parameterized) model Y ~ N(ab, 1), a normal distribution with mean ab. We can only identify the product, ab, not the individual parameters. In this case, anything along the line (ridge) ab=mu is a point of maximum likelihood. $\endgroup$ – jrb Sep 15 '20 at 22:43
-1
$\begingroup$

in micro,it means the instrumental variables used for identification is weakly correalted with the endogenous varibles. as for macro,i guess it means the parameter that are weakly identified can hardly correspond to an unique data process(as shown in your question)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.