My question is on Bayesian regression problems, where I want to formulate my problem as:
Find the posterior pdf of parameters $\theta$, where model $f_\theta$ relates observations $y_i = f_\theta(\theta,t_i)+\epsilon$ to the parameters $\theta$ and modeling errors $\epsilon$. Estimate $\theta$ while including prior information on $\theta$.
I emphasize that model $f_\theta(\theta,t_i)$ is not a dynamical model although time instants $t_i$ are mentioned! Function $f_\theta$ does not propagate $\theta$ through time, but is a black box computation that integrates IMU rates (accelerations and rotation rates) over time to produce a flight trajectory lasting several minutes (atmospheric entry on Mars). The IMU integration starts from initial trajectory conditions $\theta$ (position, velocity, attitude). The outputs of $f_\theta$ are several trajectory variables at times $t_i$ which I want to compare to independent observations, e.g. IMU integrated altitude after 60 seconds to be compared with a radar measurement at that time. The entire data record of the trajectory is available already. The objective is to optimize initial trajectory conditions $\theta$ to produce a good match between the IMU integrated trajectory, and independent measurements $y_i$ at times $t_i$. All measurements $y_i$ are to be used at once, meaning that in this "fitting problem" the time values are just known X-coordinates and time propagation of states or uncertainties is not required. The model $f_\theta$ just computes some values to be compared with measurements, which happen to have occurred at different times during flight.
Popular Bayesian methods are MCMC and Laplace (approximate). I am unfamiliar with these, and worried about their computational cost. However, I do know something about Kalman filtering (KF) and thought of reformulating this problem as a Kalman update step.
In KF the function $f_\theta$ would be a measurement model as it computes predicted observations (e.g. altitude at 60 s) from a prior state estimate, in this case parameter $\theta$ which represents the initial trajectory conditions. Predicted trajectory observations (such as IMU altitudes) are to be compared with actual measurements $y_i$ that are independent of the predicted observations (e.g. from radar instead of IMU). My hope is that this Kalman update step (where a gain is computed to correct $\theta$) will improve the estimate of initial trajectory state $\theta$ and provide a posterior pdf that is based on:
- measurement uncertainties $\sigma_{y_i}$ (zero-mean Gaussian)
- prior knowledge of initial trajectory conditions $\mu_{\theta_{prior}}$ and $\sigma_{\theta_{prior}}$
To me, this seems identical to a Kalman update step where the prior is a propagated state, but here is the $\theta$ vector with prior mean and covariance. An issue with my approach could be the inclusion of $\epsilon$, because as far as I know the Kalman filter does not allow to specify uncertainty on the measurement model directly, only the covariance of the prior is propagated through the (deterministic) measurement model.
Advice on other methods, such as MCMC or Laplace, are more than welcome of course. Or a MATLAB toolkit (Mathworks or 3rd party) I could use to solve this Bayesian regression problem?