Explain Root Mean Square Error to non-technical audience

My company is in the process of switching equipment from one vendor to another. We measured several metrics from the existing and new equipment and compared the time series. The ideal is to have no difference between the old and new equipment. For that, we use Root Mean Square Error (RMSE) to measure the difference.

We are to present the findings to upper management and employees at the plant, many of whom have no background in statistics. How should I explain the concept of RMSE to those audiences? If series X and Y have an RMSE of 5, how should I explain that 5 in layman's terms?

• Say you have measurements with one equipment, when you compare to a gold standard, say specimenes with very precise measurements taken as the truth (and the lab didn't know the thruth.) Then the RMSE is $$\sqrt{ \frac1{n}\sum_{i=1}^n (x_i-\text{gold}_i)^2 }$$ so is the standard deviation of the measurements errors. So, if the measurement errors $$x_i-\text{gold}_i$$ is normalishly distributed, around 95% of the measurement errors will be in $$\pm 2\cdot\text{RMSE}$$. You should also show some plots with your actual data. Say the equipment measures some length in mm, then your example value of 5 means a typical error of around 5 mm. (With a gold standard, your measurements can also be used to see if the measurements are typically to low or too high, or the bias. The RMSE does not measure bias alone, but bias is a component of it. See Bias-Variance decomposition derivation.)
• Without a gold standard, say you have two series of measurements, using the two equipments on the same specimenes. Then the RMSE is $$\sqrt{ \frac1{n}\sum_{i=1}^n (x_i-y_i)^2 }$$ is a measure of the typical difference between the measurement results. It should be given together with a plot of the differences. Again, the squaring forgets about the sign of the difference, which could be important too.