Wald test and Likelihood ratio test, where do the confidence intervals on the regression coefficients come from?

So I'm trying to build my own Wald test and likelihood ratio test code within a machine learning pipeline. I can get the final fitted logistic regression coefficients from liblinear. I'm coding in MATLAB.

How would I get the variance of the regression coefficients and also the confidence intervals of the regression coefficients? Clearly for a variance and confidence interval, you need a sample of multiple sets of coefficients. But I thought you only get a single set of coefficients at the end of the log likelihood optimization.

Basically trying to replicate the results in the following link http://www.ats.ucla.edu/stat/mult_pkg/faq/general/nested_tests.htm

• You can estimate the variance-covariance matrix for the coefficient estimates from the Hessian of the log-likelihood; you don't need "multiple sets" of coefficients. – Glen_b Jun 5 '14 at 23:26
• So for the Wald test, \hat\theta_i is the MLE estimate of my logistic regression coeff \theta_i (which would be the values of \theta_i that I am using for my LR model found through some descent algorithm). \theta_0 = 0, since I'm comparing against the null hypothesis that this coefficient has no effect. var(\hat\theta) is the diagonal of the Hessian matrix of the LR log-likelihood function. Is that right? – user2422566 Jun 6 '14 at 20:17
• Where the Hessian is available, the estimate of the covariance matrix is usually obtained as the inverse of the Hessian. [With Fisher scoring, I think there's an alternative approach that can be used.] – Glen_b Jun 7 '14 at 2:38
• Thanks! Also I found some good links here -- public.iastate.edu/~mervyn/stat580/Notes/s09mle.pdf data.princeton.edu/wws509/notes/a1.pdf – user2422566 Jun 9 '14 at 18:22