We're trying to use a Gaussian process to model h(t) -- the hazard function -- for a very small initial population, and then fit that using the available data. While this gives us nice plots for credible sets for h(t) and so on, it unfortunately is also just pushing the inference problem from h(t) to the covariance function of our process. Perhaps predictably, we have several reasonable and equally defensible guesses for this that all produce different result.

Has anyone run across any good approaches for addressing such a problem? Gaussian-process related or otherwise?

  • 5
    $\begingroup$ I think that if you ask a question that can be understood by people that don't know what the hazard function is and if you elaborate a bit more on what you do (how do you estimate the parameter of you'r gausian process, how do you use the gaussian process at the end) you will increase the chances to get an answer and this will be an added value for stats.stackexchange :) $\endgroup$ Jul 24 '10 at 11:01
  • $\begingroup$ when you say very small how small are you talking about? Unfortunately is is common to see people using sophisticated methods applied to a handful of data. If the problem is not having a good study to start with one should look at alternatives and make a case for a better study or data collection $\endgroup$
    – tosonb1
    Sep 9 '10 at 23:58

This is probably a stupid answer (I am new here), but if you want to estimate the hazard function from observations of an initial population that slowly died away (i.e. had events and then were censored), isn't that what the Nelson-Aalen estimator was built to do?

We could have another conversation about the reliability of the available classical confidence intervals -- my understanding is that there basically do not exist functioning exact confidence intervals that guarantee their coverage even over small sample sizes, since such an interval would need to work over all distributions of censoring time. (Maybe the problem is simpler when individuals are always censored exactly after their first event.) And mapping out the coverage of an approximate interval precisely would take work.

But if you just need a point estimate, the Nelson-Aalen estimator seems to do the trick. (It's a lot like the Kaplan-Meier estimate for the survival function...)

If you want to calculate an a posteriori distribution on a whole family of possible hazard functions, and your prior is that they are drawn from the Gaussian processes with certain statistics, can you explain further what the difficulty is? If there isn't agreement on the covariance matrix, then that needs to be part of the prior -- that the covariance matrix is drawn from some distribution. You're not going to get around having to state a prior if the goal is a posterior.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.