Thanks for your suggestions.Actually i have the following model that i explains to you.

Suppose i have observations structure like $$\begin{align} y_1 &=&v_1+e_1 \\ y_2 &=&v_2+e_2 \\ y_3&=&v_1^2-v_1v_2(\cos(x_1-x_2)+\sin(x_1-x_2))+e_3 \end{align}$$, and so on......

Every error term assume to follow normal distribution with mean zero but different variance. In model form above observations can be written as

Y=XB+e.(heterscedastic and nonlinear model) Here v1,v2 x1,x2....are unknown values and surely model is nonlinear in unknown parameters. We use Newton type method to estimates the unknown parameters. Guass Newton method work well to find estimates in this problem. I can find Hessian matrix or fisher Information matrix by this method that can be used to find standard errors of the estimates.

Now my question is related to confidence intervals for unknown parameters. i construct the confidence intervals for unknown parameters by using the standards that are calculated from Hessian matrix .I construct individual confidence intervals for each parameter by using t statistics?(Wald Type).When i simulate this model by knowing the actual values of parameters i found that coverage probabilities are very low special for unknown parameters angles.

Appreciate if you could suggest the technique to tackle this problem.

  • $\begingroup$ What model is it? There is a very general class of models that allows you to construct a theoretically sound (albeit valid only asymptotically) confidence interval by inverting a Wald test, or by using the delta method $\endgroup$ Jan 13, 2015 at 16:57
  • $\begingroup$ Dear Harvey Motulsky:Could you please comments now to the problem $\endgroup$
    – zahid khan
    Jan 18, 2015 at 14:55

1 Answer 1


Sounds like you want to do a simulation. In that case you know the population parameter and the proportion of confidence intervals that include that true parameter value is your coverage probability.

To quantify the the comment by @HarveyMotulsky: In a given scenario (sample size, number, distribution and correlation of independent/explanatory/right-hand-side/x-variables, strength of effects, marginal variance of the dependent/explained/left-hand-side/y-variable, ...) you would expect about 5% of the replications to fall outside your 95% confidence interval. So if you have 100 replications, you would expect to base your estimate of the coverage on only 5 replications. That is obviously not enough. For 95% confidence intervals I tend to use 20,000 replications; that way I expect to base my estimate of the coverage on 1,000 replications in which the true hypothesis is rejected.

Once you have done so for one scenario, you change one of the characteristics (e.g. change the sample size), and do the simulation again. You continue doing that until you have a good idea of under what conditions your test fails and under what conditions your test is fine.

You would not expect a simple conclusion like "the confidence interval is right", because any statistic will fail if you use it on extreme enough data. Instead, the purpose for a simulation study should be to find the conditions under which the test performs acceptable, the conditions under which the test will not perform acceptable, and the "grey zone" where the user should be extra careful when using this test.

  • 1
    $\begingroup$ For the sake of others reading this: You need to run lots (thousands) of simulations. Running one simulation won't help. $\endgroup$ Jan 13, 2015 at 16:08
  • 1
    $\begingroup$ @Harvey A very worthwhile clarification; a lot of people (myself included) say 'a simulation' as shorthand for something like 'a simulation study' ... which will itself have many, many sets of simulated data in it. $\endgroup$
    – Glen_b
    Jan 13, 2015 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.