Error Propagation and Error Partition

Given a a general form of a model $F$ ( which could be an explicit formula or set of formulas) with its input $x$ and output $y$.

Suppose that the input $x$ has an error $\delta x$, so that given $\tilde{x}=x+\delta x$ , and suppose also that the model $F$ is not exact. Then the output $y$ is surely infected by error, said the obtained output is $\tilde{y}=y+\delta y$.

My question is : How could one partition the output error into error coming from $F$ and input error ? In other words, How could we write the error $\delta y$ as two parts, first part coming from the error of $x$ and the second part coming from the error of $F$?

Does anay one have an idea ? Any help would be highly appretiated.

• Is this homework? In general explanations how to do error propagation can be found in most text books about data analysis. Do you have trouble applying these? And please remove the backpropagation tag -- that is about learning/updating rules in neural networks. – cherub Sep 27 '16 at 13:20
• @cherub No it is not a homework, it is a self thinking. Infact I did differetn researches about the back propagation of the error, but I haven't find a general case analysis, so would you please lead me to a clear book, article, or any source in which I can find what I am searching for . – Nizar Sep 28 '16 at 7:34

A possible answer is that in fact the wiki article already sums it up pretty well: https://en.wikipedia.org/wiki/Propagation_of_uncertainty.

In general you need the partial derivatives, or in the multidimensional case the Jacobian (matrix). If you know the function, this is rather straightforward an exercise in calculus. In case of a linear function, the derivatives become trivial.

In the general case of an arbitrary function, you can also chose from several other options. The article lists five variants; my practical knowledge extends to two of these. The accuracy and feasibility depend on the type of function and the kind of approximation you need.

1) Series evolution: you can do the Taylor expansion and stop at your choice of order. This introduces a shift in the expectation value the propagated error w.r.t the true error (of an infinite series) -- it becomes biased. Given a well behaved function, this might be negligible.

2) Monte-Carlo simulation. Has the benefit that you might not even have to know the actual analytical (if existing) form of your function. But this becomes an own branch of research, depending on the function. Keywords here are efficiency and accuracy of the calculation.

The other methods are more specialized, mostly to problems that can't be well approximated by the two above.

So, concerning your original question: to disentangle the uncertainty $\delta y$, simply write down the full Taylor expansion term. For the first order (which is sufficient in most cases) have a look at the wiki page.

• In the wiki page, they are explaining how the uncertainty propagates from the input x to the output , through a model f. Thus they represent the output uncertainty in terms of the input uncertainty. However , my question was, what if even the given f is not the true exact model? In this case expressing the uncertainty of y in terms of the uncertainty of x , ignoring the effect of f, would give non realistic results. – Nizar Sep 28 '16 at 14:46
• @Nizar. OK, I understand, at least partially. I don't really understand the specific case, because any inherent uncertainty in the model you would need to model itself -- and then do the variations as described above (e.g. MC sim based) to quantify that particular uncertainty. In the general case you'll probably have a correlation between the uncertainties -- so disentangling might become difficult. But it's hard to say how to approach this problem in particular. In general I would think that the approach in my answer would be the clearest starting point. [tbc] – cherub Sep 29 '16 at 18:00
• [cont'd] The idea to have a model (function) with an uncertainty seems somewhat strange, although I can't put my finger on it. The determination of an error of an error (input uncertainty folded with the model uncertainty) does not make sense. I'll continue to figure out, why that seems wrong. – cherub Sep 29 '16 at 18:03