Errors-in-variables models are defined as:
regression models that account for measurement errors in the independent variables. In contrast, standard regression models assume that those regressors have been measured exactly, or observed without error; as such, those models account only for errors in the dependent variables, or responses.
Whether data is experimental or computational (e.g. via physics-based simulation), uncertainty in the independent variables is common. This is particularly relevant in materials science where uncertainty exists in the processing, structure, and property measurements. Additionally, this uncertainty is not necessarily constant for all parameters (a temperature sensor might have higher uncertainty at the limits of the temperature range, various aspects of a material's structure may have been determined by separate instruments, property measurements are data-mined from several sources, etc.). See also Uncertainty Quantification. I'm sure this is relevant for other disciplines too (e.g. physics, chemistry, biology, engineering).
What I'm not looking for
- linear regression models (e.g. orthogonal regression/total least squares OrthogonalLineFit, linortfitn(), odr(), odregress()).
- response-uncertainty-only regression models
- Nonlinear parametric fitting with a specified vector of response uncertainties using NonlinearModelFit
- Non-parametric fitting with a specified scalar response uncertainty using fitrgp(). This allows for the specification of a vector of "smoothness lengths", one per predictor, but not for specifying uncertainty of predictors
What I am looking for
A method that caters to multidimensional, non-parametric regression with propagated measurement uncertainty in predictors and responses (i.e. uncertainty propagation, not just weighting the points) and preferably software that goes along with it (Mathematica, MATLAB, Python, R, Stan, etc.). Multidimensional refers to the predictors (i.e. responses are scalar).
Ideally, it would be something that is explained and can be performed via:
mdl = some_function(X, Xsd, y, ysd) ypred, ysd2, cov = mdl(X2, Xsd2)
- Matrix/Array of predictors (
- Matrix/Array of predictor uncertainties (
- Vector of response values (
- Vector of response uncertainties (
- Matrix/Array of new predictors (
- (Optional) Matrix/Array of new predictor uncertainties (
- Vector of new responses (
- Vector of new response uncertainties (
- (Optional) predictor covariance matrix (
- This seems to be supported through BoTorch, a package based in Python (see tutorial), and to later be exposed in Meta's Adaptive Experimentation platform. May eventually try this and flesh it out into an answer. If someone else would like to try it out and provide a MWE, this is also welcome.
- SE: Non-linear fitting with uncertainty in dependent and independent variable
- Errors-in-variables multivariate polynomial regression (R)
- Including model uncertainty in non-linear least squares minimization
- How do I specify the uncertainty of a parameter when using NonlinearModelFit?
- How to add uncertainty to your neural network
- BLiTZ — A Bayesian Neural Network library for PyTorch