# How to compute variance of Cox model coefficient estimate using Fisher information?

We have Cox proportional hazards model: $$\lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$ where $$\boldsymbol \beta$$ and $$\boldsymbol x$$ are vectors. To make it simple, lets say that there are only 3 prognostic variables.

Say that we measured $$(T_i,\delta_i,X_i)$$ on $$n$$ subjects, where

$$T_i$$ are measured times, censored or not,
$$\delta_i$$ are indicators of censoring (1 = event, 0 = censoring),
$$X_i$$ is a vector of prognostic variables.

Let $$t_1,...,t_k$$ be ordered, distinct times of events, so that at any $$t_i$$ only one event occurs. Denote by $$R(t) = \{i: t_i \geq t \}$$ a set of subject still at risk at $$t$$.

We then have logarithm of partial likelihood function: $$l(\beta) = \sum_{i=1}^n \delta_i \left[\beta'x_i - \text{ln}\left(\sum_{j\in R(t_i)}exp(\beta'x_j) \right) \right],$$ from which we get maximum likelihood estimators $$\hat\beta_1,\hat\beta_2,\hat\beta_3.$$ For each $$k\in \{1,2,3\}$$ it follows that $$\frac{\hat \beta_k - \beta_k}{se(\hat \beta_k)}\sim N(0,1)$$ and that variance of estimated parameter $$\hat \beta_k$$ can be approximated using Fisher information: $$\text{var}(\hat\beta_k) \approx \left(-\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right)^{-1}.$$

Variance of estimated parameter comes from Fisher information, saying that for random variable $$X$$ and unknown parameter $$\theta$$ upon which the probability of $$X$$ depends: $$I(\theta) = -\text{E}\left(\frac{\partial^2}{\partial\theta^2}log(f(X|\theta))\right),$$ $$\text{var}(\hat \theta) \geq I(\theta)^{-1}$$ where $$f(X|\theta)$$ is probability density function for $$X$$ conditional on $$\theta$$.

My questions are:

1. Why is approximation for $$\text{var}(\hat \beta_k)$$ without expected value? I would expect it to be $$\text{var}(\hat\beta_k) \approx \left(-\text{E}\left[\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right]\right)^{-1}.$$ 2. Can we use $$l(\boldsymbol \beta)$$ instead of log pdf when applying Fisher information?
3. How could we write $$f(X|\theta)$$ in case of Cox model?

Thank you!

I will first try to answer your first question. To show that we can estimate the variance $$\text{Var}(\hat{\theta}_{n})$$ of maximum likelihood estimator $$\hat{\theta}_{n}$$ with second log likelihood derivative, we can try to derive asymptotic relationship between second log likelihood derivative $$l''(\theta)$$ and Fisher information $$I_{n}(\theta)^{-1}$$. The negative of second log likelihood derivative $$-l''(\theta)$$ is called observed Fisher information.

Let $$X_{1}, ..., X_{n}$$ form a random sample from a distribution for which the p.d.f. (or p.f.) is $$f(x \mid \theta)$$ then the likelihood is a product $$L_{n}(\theta) = \prod_{i = 1}^{n} f(x_{i} \mid \theta)$$ then the log likelihood and its derivatives are the sums \begin{align*} l_{n}(\theta) &= \sum_{i = 1}^{n} \log f(x_{i} \mid \theta) \\ l'_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial}{\partial \theta} \log f(x_{i} \mid \theta) \\ l''_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta) \end{align*}

Fisher information in the entire sample is defined as $$I_{n}(\theta) = -E_{\theta}\left[ l''_{n}(\theta) \right]$$ Since data are i.i.d.\ we also have $$I_{n}(\theta) = n \cdot I_{1}(\theta).$$

Let $$\hat{\theta}_{n}$$ be the maximum likelihood estimator (MLE) of $$\theta$$. Based on the asymptotic normality of MLE the distribution of $$\hat{\theta}_{n}$$ is approximately $$\mathcal{N}(\theta, I_{n}(\theta)^{-1})$$ so we can estimate the variance $$Var(\hat{\theta}_{n})$$ with Fisher information $$I_{n}(\theta)^{-1}$$.

Since $$n$$ random variables $$X_{1}, ..., X_{n}$$ are i.i.d., the $$n$$ random variables $$l''(X_{1} \mid \theta), ..., l''(X_{n} \mid \theta)$$ are also i.i.d., where we define $$l''(x \mid \theta) = \frac{\partial^2}{\partial \theta^2} \log f(x \mid \theta)$$.

Then dividing above equation by $$n$$ we get average of i.i.d. random variables $$\frac{l''_{n}(\theta)}{n} = \frac{1}{n} \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta)$$ and we can calculate the expectation $$E_{\theta} \left[ l''_{1}(\theta) \right] = -I_{1}(\theta)$$

and now we can apply (weak) law of large numbers (LLN) to get convergence in probability $$\frac{l''_{n}(\theta)}{n} \overset{p}{\to} -I_{1}(\theta).$$ which is why we can approximate $$I(\theta)$$ with the second derivative of the log likelihood $$l''(\theta)$$.

To show this in your case of Cox regression model in a more rigorous way we need to generalize this from random variables to martingales.

• Thank you for answer! So the answer on my first question is that we may skip expected value because convergence in probability. Commented Mar 28, 2020 at 18:32