We have Cox proportional hazards model: $$ \lambda(t,x) = \lambda_0(t)exp(\boldsymbol \beta'\boldsymbol x),$$ where $\boldsymbol \beta$ and $\boldsymbol x$ are vectors. To make it simple, lets say that there are only 3 prognostic variables.

Say that we measured $(T_i,\delta_i,X_i)$ on $n$ subjects, where

$T_i$ are measured times, censored or not,
$\delta_i$ are indicators of censoring (1 = event, 0 = censoring),
$X_i$ is a vector of prognostic variables.

Let $t_1,...,t_k$ be ordered, distinct times of events, so that at any $t_i$ only one event occurs. Denote by $R(t) = \{i: t_i \geq t \}$ a set of subject still at risk at $t$.

We then have logarithm of partial likelihood function: $$l(\beta) = \sum_{i=1}^n \delta_i \left[\beta'x_i - \text{ln}\left(\sum_{j\in R(t_i)}exp(\beta'x_j) \right) \right],$$ from which we get maximum likelihood estimators $\hat\beta_1,\hat\beta_2,\hat\beta_3.$ For each $k\in \{1,2,3\}$ it follows that $$\frac{\hat \beta_k - \beta_k}{se(\hat \beta_k)}\sim N(0,1)$$ and that variance of estimated parameter $\hat \beta_k$ can be approximated using Fisher information: $$\text{var}(\hat\beta_k) \approx \left(-\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right)^{-1}.$$

Variance of estimated parameter comes from Fisher information, saying that for random variable $X$ and unknown parameter $\theta$ upon which the probability of $X$ depends: $$I(\theta) = -\text{E}\left(\frac{\partial^2}{\partial\theta^2}log(f(X|\theta))\right),$$ $$\text{var}(\hat \theta) \geq I(\theta)^{-1}$$ where $f(X|\theta)$ is probability density function for $X$ conditional on $\theta$.

My questions are:

1. Why is approximation for $\text{var}(\hat \beta_k)$ without expected value? I would expect it to be $$\text{var}(\hat\beta_k) \approx \left(-\text{E}\left[\frac{\partial^2}{\partial \beta_k^2}l(\beta)\right]\right)^{-1}.$$ 2. Can we use $l(\boldsymbol \beta)$ instead of log pdf when applying Fisher information?
3. How could we write $f(X|\theta)$ in case of Cox model?

Thank you!


1 Answer 1


I will first try to answer your first question. To show that we can estimate the variance $\text{Var}(\hat{\theta}_{n})$ of maximum likelihood estimator $\hat{\theta}_{n}$ with second log likelihood derivative, we can try to derive asymptotic relationship between second log likelihood derivative $l''(\theta)$ and Fisher information $I_{n}(\theta)^{-1}$. The negative of second log likelihood derivative $-l''(\theta)$ is called observed Fisher information.

Let $X_{1}, ..., X_{n}$ form a random sample from a distribution for which the p.d.f. (or p.f.) is $f(x \mid \theta)$ then the likelihood is a product $$ L_{n}(\theta) = \prod_{i = 1}^{n} f(x_{i} \mid \theta) $$ then the log likelihood and its derivatives are the sums \begin{align*} l_{n}(\theta) &= \sum_{i = 1}^{n} \log f(x_{i} \mid \theta) \\ l'_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial}{\partial \theta} \log f(x_{i} \mid \theta) \\ l''_{n}(\theta) &= \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta) \end{align*}

Fisher information in the entire sample is defined as $$ I_{n}(\theta) = -E_{\theta}\left[ l''_{n}(\theta) \right] $$ Since data are i.i.d.\ we also have $$ I_{n}(\theta) = n \cdot I_{1}(\theta). $$

Let $\hat{\theta}_{n}$ be the maximum likelihood estimator (MLE) of $\theta$. Based on the asymptotic normality of MLE the distribution of $\hat{\theta}_{n}$ is approximately $$ \mathcal{N}(\theta, I_{n}(\theta)^{-1}) $$ so we can estimate the variance $Var(\hat{\theta}_{n})$ with Fisher information $I_{n}(\theta)^{-1}$.

Since $n$ random variables $X_{1}, ..., X_{n}$ are i.i.d., the $n$ random variables $l''(X_{1} \mid \theta), ..., l''(X_{n} \mid \theta) $ are also i.i.d., where we define $l''(x \mid \theta) = \frac{\partial^2}{\partial \theta^2} \log f(x \mid \theta)$.

Then dividing above equation by $n$ we get average of i.i.d. random variables $$ \frac{l''_{n}(\theta)}{n} = \frac{1}{n} \sum_{i = 1}^{n} \frac{\partial^2}{\partial \theta^2} \log f(x_{i} \mid \theta) $$ and we can calculate the expectation $$ E_{\theta} \left[ l''_{1}(\theta) \right] = -I_{1}(\theta) $$

and now we can apply (weak) law of large numbers (LLN) to get convergence in probability $$ \frac{l''_{n}(\theta)}{n} \overset{p}{\to} -I_{1}(\theta). $$ which is why we can approximate $I(\theta)$ with the second derivative of the log likelihood $l''(\theta)$.

To show this in your case of Cox regression model in a more rigorous way we need to generalize this from random variables to martingales.

  • $\begingroup$ Thank you for answer! So the answer on my first question is that we may skip expected value because convergence in probability. $\endgroup$ Mar 28, 2020 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.