2
$\begingroup$

I have uniformly sampled datasets from two acquisition circuits: an old model which has become difficult (and expensive) to procure parts to continue manufacturing, and a new version using modern microelectronics. Paired datasets have been produced by measuring the same phenomenon using the same transducers and swapping the acquisition circuits. (Each experiment produces a dataset: I have pairs of datasets, not datasets of pointwise paired experiments)

The paired datasets ought to be be the same scale and alignment along the independent variable, but in practice, tolerances are causing non-negligible mismatch. I'm looking for a metric to measure agreement between the old and new systems.

enter image description here

The classic is to assume alignment and find mean absolute value or rms of the difference in dependent variable, evaluated at the same index.

$$ \left ( \sum \left| x(f) - y(f) \right|^p \right)^{1/p} $$

But that breaks down when the independent variable is misaligned.

For regression, one can use "orthogonal least-squares", and, based on some distance metric on points, calculate a norm of the distances between samples and nearest approach of the regression curve.

But here I don't have a parametric curve to compare against, I have another sampled dataset.

Also, if the scaling were identical and only the position had a small shift, one could use apply a translation optimized by maximizing auto-correlation, and then measure distance in only the dependent variable. But I suspect small deviations also in scaling along the independent variable.

Is there a common metric used for distance between one sampled dataset and another dataset, also sampled?

I'm considering a "distance to nearest neighbor" approach, requiring all points from both datasets to be used, to avoid an undesirable asymmetry. Any expected problems with that approach?

$\endgroup$
2
  • 1
    $\begingroup$ Dynamic time warping may be of interest. $\endgroup$ Nov 8, 2023 at 20:39
  • 1
    $\begingroup$ @StephanKolassa: That looks very applicable, thanks. It looks very like Levenshtein distance, with insertion and deletion operations, only with soft- instead of hard- decision matching. I'll explore some of the references from there as well. Thanks for the pointer! $\endgroup$
    – Ben Voigt
    Nov 8, 2023 at 20:49

1 Answer 1

2
$\begingroup$

Dynamic time warping (DTW), as suggested by @StephanKolassa, is an excellent idea with lots of available tools. Perhaps a more modern, foundational approach would be to apply functional data analysis (FDA). One such approach, called elastic FDA, provides tools which can be viewed as a continuous extension of DTW. These slides provide a nice starting place, if you're interested in learning more.

Note: I am not an FDA person myself, but I at least partially buy into their advantages over classical discrete approaches like DTW. The argument for FDA usually goes: develop the theory on function spaces, not vectors, and discretize only at the last step: implementation. This is summed up in the short quote by the father of FDA, "Discretize as late as possible" -Ulf Grenander.


Broadly, the strategy is

  1. Try to align the functions

  2. (A) Measure amplitude distance on aligned functions

  3. (B) Measure phase distance on aligned functions based on how much "work" was needed to align the functions.

Let's walk through a simple example with the fdasrvf package (in R).

# Generate data
N <- 100
t <- sort(runif(N))
f1 <- sin(4*pi*t) * t
f2 <- sin(4*pi*(t-1/20)) * t^1.1

plot(t, f1, type='l',
     main="Original misaligned data")
lines(t, f2, col='orange')

enter image description here

Next, I will use the elastic.distance function to get some measures of the distance between these two functions

fdasrvf::elastic.distance(f1, f2, t)

which returns

$Dy
[1] 2.712711

$Dx
[1] 0.3922354

The $D_y$ term is called the amplitude distance and $D_x$ is called the phase distance. Essentially, $D_y$ is what you want if the functions are aligned and $D_x$ tells you how hard we had to work to align the functions.

To break this down a little farther, let's see what the elastic FDA is doing. First, it attempts to align the functions, yielding something like:

ff <- cbind(f1, f2)
fda <- fdasrvf::align_fPCA(ff, t, lambda=0)

plot(t, fda$fn[,1], type='l', main="Aligned Data")
lines(t, fda$fn[,2], col='orange')

enter image description here

Now that the functions have been "phase-aligned", the usual amplitude distance is well defined and meaningful. In order to get these aligned functions, we apply a warping function, so that $f_\text{align}(t) = \gamma(f(t))$. These warping functions look like:

plot(t, fda$gam[,1], type='l', main="Warping functions")
lines(t, fda$gam[,2], col='orange')
abline(0, 1, lwd=2, col='red')

enter image description here

How far these functions differ from the identity line tells us how hard we had to work to align them. This is directly related to $D_x$, the phase distance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.