1
$\begingroup$

I have observed a vector of quantities $\vec y$. I wish to use these to constrain a vector of initial conditions $\vec x$ that are related to $\vec y$ through a non-linear (numerically evaluated) function $f$, i.e. I want to find $$\hat x = \underset{\vec x}{\arg\min} \; \left[ f(\vec x) - \vec y \right]^2.$$ However, $\vec y$ has been measured with uncertainties $\vec \sigma$. Therefore, I should minimize something like a reduced $\chi^2$: $$\hat x = \underset{\vec x}{\arg\min} \; \sum {\frac{\left[ f(\vec x) - \vec y \right]^2}{\sigma^2}}.$$ However, I fear the following two situations:

  1. $f$ is nonlinear, so two elements of $f(\vec x)$ could provide identical information, and then my minimization would be biased towards that redundant information. The values $\vec y$ have been measured with covariance, but this covariance only accounts for the measurement error and does not account for the covariance in the model $f$.

  2. If one of the variables is measured much more precisely than the other variables, then it will effectively be the only thing being fit by the minimization procedure.

Are there any ways around 1 and 2?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ You're more or less describing (possibly weighted) nonlinear least squares, but your paradigm is not clear. Please distinguish your data from your parameter (vector) estimation problem. Do you have m data points of right hand side values with corresponding left hand side (y) values? If so, what's changing between the input points? For any given data point, is y a scalar (output), or is it a vector output? Are the y's uncorrelated across data points? Do you have different values of $\sigma$ for different data points? What do you mean by covariance in $f$. Etc. $\endgroup$ – Mark L. Stone May 31 '16 at 22:04
  • $\begingroup$ If your $\mathbf{y}$ is a vector, then you may want to do something like inverse variance weighting $\mathrm{argmin}_{\mathbf{x}} \left(\mathbf{f}(\mathbf{x}) - \mathbf{y} \right)' \Sigma^{-1} \left(\mathbf{f}(\mathbf{x}) - \mathbf{y} \right) $ $\endgroup$ – Matthew Gunn May 31 '16 at 22:51
  • $\begingroup$ You might want to add some regularization (e.g. Tikhonov or L1) to avoid overfitting. Other than that, you have a nonlinear optimization problem. If you can differentiate f easily, you can use Gauss Newton. Let me know if you need more clarifications. $\endgroup$ – Yair Daon Jun 4 '16 at 3:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.