How to compute variance of Cox model coefficient estimate using Fisher information?

Question

We have Cox proportional hazards model: $$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$ where $\boldsymbol \beta$ and $\boldsymbol x$ are vectors. To make it simple, lets say that there are only 3 prognostic variables.

Say that we measured $(T_i,\delta_i,X_i)$ on $n$ subjects, where

$T_i$ are measured times, censored or not,
$\delta_i$ are indicators of censoring (1 = event, 0 = censoring),
$X_i$ is a vector of prognostic variables.

Let $t_1,...,t_k$ be ordered, distinct times of events, so that at any $t_i$ only one event occurs. Denote by $R(t) = \{i: t_i \geq t \}$ a set of subject still at risk at $t$.

We then have logarithm of partial likelihood function: $$l(\beta) = \sum_{i=1}^n \delta_i \left[\beta'x_i - \text{ln}\left(\sum_{j\in R(t_i)}exp(\beta'x_j) \right) \right],$$ from which we get maximum likelihood estimators $\hat\beta_1,\hat\beta_2,\hat\beta_3.$ For each $k\in \{1,2,3\}$ it follows that $$\frac{\hat \beta_k - \beta_k}{se(\hat \beta_k)}\sim N(0,1)$$ and that variance of estimated parameter $\hat \beta_k$ can be approximated using Fisher information: $$\text{var}(\hat\beta_k) \approx \left(-\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right)^{-1}.$$

Variance of estimated parameter comes from Fisher information, saying that for random variable $X$ and unknown parameter $\theta$ upon which the probability of $X$ depends: $$I(\theta) = -\text{E}\left(\frac{\partial^2}{\partial\theta^2}log(f(X|\theta))\right),$$ $$\text{var}(\hat \theta) \geq I(\theta)^{-1}$$ where $f(X|\theta)$ is probability density function for $X$ conditional on $\theta$.

My questions are:

1. Why is approximation for $\text{var}(\hat \beta_k)$ without expected value? I would expect it to be $$\text{var}(\hat\beta_k) \approx \left(-\text{E}\left[\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right]\right)^{-1}.$$ 2. Can we use $l(\boldsymbol \beta)$ instead of log pdf when applying Fisher information?
3. How could we write $f(X|\theta)$ in case of Cox model?

Thank you!

Answer 1

I will first try to answer your first question. To show that we can estimate the variance $\text{Var}(\hat{\theta}_{n})$ of maximum likelihood estimator $\hat{\theta}_{n}$ with second log likelihood derivative, we can try to derive asymptotic relationship between second log likelihood derivative $l''(\theta)$ and Fisher information $I_{n}(\theta)^{-1}$. The negative of second log likelihood derivative $-l''(\theta)$ is called observed Fisher information.

Let $X_{1}, ..., X_{n}$ form a random sample from a distribution for which the p.d.f. (or p.f.) is $f(x \mid \theta)$ then the likelihood is a product $$ L_{n}(\theta) = \prod_{i = 1}^{n} f(x_{i} \mid \theta) $$ then the log likelihood and its derivatives are the sums \begin{align*} l_{n}(\theta) &= \sum_{i = 1}^{n} \log f(x_{i} \mid \theta) \\ l'_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial}{\partial \theta} \log f(x_{i} \mid \theta) \\ l''_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta) \end{align*}

Fisher information in the entire sample is defined as $$ I_{n}(\theta) = -E_{\theta}\left[ l''_{n}(\theta) \right] $$ Since data are i.i.d.\ we also have $$ I_{n}(\theta) = n \cdot I_{1}(\theta). $$

Let $\hat{\theta}_{n}$ be the maximum likelihood estimator (MLE) of $\theta$. Based on the asymptotic normality of MLE the distribution of $\hat{\theta}_{n}$ is approximately $$ \mathcal{N}(\theta, I_{n}(\theta)^{-1}) $$ so we can estimate the variance $Var(\hat{\theta}_{n})$ with Fisher information $I_{n}(\theta)^{-1}$.

Since $n$ random variables $X_{1}, ..., X_{n}$ are i.i.d., the $n$ random variables $l''(X_{1} \mid \theta), ..., l''(X_{n} \mid \theta) $ are also i.i.d., where we define $l''(x \mid \theta) = \frac{\partial^2}{\partial \theta^2} \log f(x \mid \theta)$.

Then dividing above equation by $n$ we get average of i.i.d. random variables $$ \frac{l''_{n}(\theta)}{n} = \frac{1}{n} \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta) $$ and we can calculate the expectation $$ E_{\theta} \left[ l''_{1}(\theta) \right] = -I_{1}(\theta) $$

and now we can apply (weak) law of large numbers (LLN) to get convergence in probability $$ \frac{l''_{n}(\theta)}{n} \overset{p}{\to} -I_{1}(\theta). $$ which is why we can approximate $I(\theta)$ with the second derivative of the log likelihood $l''(\theta)$.

To show this in your case of Cox regression model in a more rigorous way we need to generalize this from random variables to martingales.