I'm using Bayesian inference to estimate the parameters of a dynamical system which is comprised of several ordinary differential equations (ODEs). I use a gaussian error model. I would like to know if I may use the variance of the residuals (model prediction-data) to estimate the variance of the errors.

To calculate the likelihood of a set of parameters given some data, I assume that measured data points are the sum of the models' prediction + some normally distributed error. Therefore, I can formulate the log-likelihood:

Let there be at the time points $t=(t_1,\ldots,t_n)$ measurement results $\vec D=(D_1,\ldots,D_n)$ and the corresponding model predicts trajectories $\vec M(t, \theta)$ which are dependent of the model's parameter values $\theta$. From the PDF of the normal distribution follows my log-likelihood: $$ D_i = M(t_i,\theta)+\varepsilon_i\quad\text{, where }\varepsilon_i\sim \mathcal{N}(0,\sigma^2)$$ $$\ln p(D|\theta)=-\frac{n}{2}\ln(2\pi)-\frac n2 \ln\sigma^2 - \frac 1{2\sigma^2}\sum_{i=1}^{|D|}(D_i-M(t_i,\theta))^2$$

I need an estimation for the variance of the error terms $\sigma^2$ before I can calculate the likelihood.

I opted to estimate $\sigma^2$ by the variance of the residuals, so $$\sigma^2(\theta) \overset{\text{estimate}}{=}\operatorname{var}(\vec D -\vec M(\vec t,\theta))$$

I recon this estimation valid because

  1. In biology, when deriving parameters (like e.g. reaction rates) from data, the residuals between the data and the fit are routinely attributed to measurement error

  2. This error model honors good fit of all data points because that leads to a small estimated error variance and hence to a high likelihood value. When assuming a constant measurement error SD in contrast, good fit of many data points may offset bad fit of some.?

However, this model implies that my measurement error ($\sigma^2$) is dependent of the parameters $\theta$ of the system I'm measuring. I think this is counterintuitive as measurement error is dependent of the measuring device and not the system itself.

Unfortunately, I did not find literature expanding on how to estimate the likelihood of ODE systems' predictions at the mathematical level I can understand. Therfore I would be glad if you could let me know if my error model is justified by reality and if it assumes something I did not mention here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.