0
$\begingroup$

I was staring at a time series and thought an interesting way to measure a variance-like value would be to treat the time series like a physical chain and the amount of slack in that chain would be kinda like variance. If the series is constant, you'd have full tension and no variance, if there are lots of wiggles, that would result in a lot of slack and higher variance. I know it's not exactly "variance" but similar. I want to know if this has a real name and show some kind of interesting results.

I calculated this as the sum of the $\ell_2$ distances between the points in the series, divided by the straight line length of the series. So if our series is written as $(x_1, y_1), (x_2, y_2) \ldots (x_n, y_n)$ and we define $p_i = (x_i, y_i)$ then our "slack" variance in general would be: $$\frac{1}{x_n-x_1}\sum_{i=1}^{n-1}\ell_2(p_{i+1}, p_i)$$

Just to make the reduction for my main question easier to see, the 2d case would be: $$\frac{1}{x_n-x_1}\sum_{i=1}^{n-1}\sqrt{(y_{i+1} - y_i)^2 + (x_{i+1} - x_i)^2}$$

And so for the 1-d case with equidistant $x$-values, which I think is probably the most common practical case, we can assume a unit distance between each $x$ WLOG to get $$\frac{1}{n-1}\sum_{i=1}^{n-1}\sqrt{(y_{i+1} - y_i)^2 + 1}$$

Which is pretty close to just the average difference between successive values in the series. For kicks, I generated a bunch of random series from different distributions and compared this statistic vs actual variance. It is correlated, but not perfect, except for Cauchy where we seem to get a perfect correlation between this and variance:

table of distribution correlations

Note that if the title doesn't have parameters, they're the default R values. Here's the code if you want to try it out:

slack <- function(x) {
  return(sum(sqrt(diff(x)^2 + 1))/(length(x)))
}

slacks <- c()
vars <- c()
for (i in 1:1000) {
  t <- rexp(1000, .100)
  slacks <- append(slacks, slack(t))
  vars <- append(vars, var(t))
}

plot(slacks, vars, main='Exp(0.1)')

Questions: Does this statistic have a name or is it in common use? Any ideas on why the Cauchy specifically fits so nice with "slack"? I know the moments for the Cauchy distribution are undefined but I can't see a good reason for this particular correlation.

$\endgroup$
1
  • $\begingroup$ cauchy variance is undefined, so the "fits so nice" that you are seeing is probably just because the scales are so different (observed variances for your cauchy variables are in magnitude 10^7) if you plotted vars and slacks on log scale you would probably they are not perfectly correlated $\endgroup$
    – bdeonovic
    Commented Aug 22, 2022 at 15:12

1 Answer 1

0
$\begingroup$

Instead of doing successive differences you could instead consider all pairwise differences (if your sample is indeed IID instead of a time series then this should be better estimate of this quantity)

quick and dirty code for something like that would be

all_pairs_slack <- function(x){
  idx <- combn(1:length(x), 2)
  mean(sqrt(apply(idx, 2, function(ii){ diff(x[ii])^2+1})))
}

I think this quantity is estimating

$$ \text{E}\left[\sqrt{(Y_1-Y_2)^2 + 1}\right] $$

Where $Y_1 \sim Y_2 \sim Y$ have the same distribution (and are independent). This is actually pretty close to variance. If you didn't have the square root and that pesky $+1$ then you would have

$$ \begin{align*} \text{E}\left[(Y_1-Y_2)^2\right] &= 2\text{Var}[Y] \end{align*} $$

See this blogpost: https://aaronjfisher.github.io/why-divide-n-1-pairwise-differences.html

As for the Cauchy phenomenon: cauchy variance is undefined, so the "fits so nice" that you are seeing is probably just because the scales are so different (observed variances for your cauchy variables are in magnitude 10^7) if you plotted vars and slacks on log scale you would see they are not perfectly correlated

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.