3
$\begingroup$

I have a summation that looks like $$\sum_{j=0}^{n-1} S_j(v_{j} - \bar{v})$$ The $S_j$ terms all have values in $[0,1]$. $v_j$ is a data value and $\bar{v}$ is the mean of all $v_j$. This expression is clearly related to the standard deviation. If $\sigma=0$ than all $v_j=\bar{v}$ so the result is $0$. But what I would like to do is relate the change in value of this expression to $\sigma$ for all values of $\sigma$. Is this possible? I have tried without luck but think that perhaps there is some property of $\sigma$ which I do not know which might help here?

$\endgroup$
  • 2
    $\begingroup$ In general no. You are right about the special case of zero SD and your sum, but SD is a root mean square and if it could be re-written as a weighted sum of deviations that would be public knowledge. I suspect at most there is an inequality lurking here, related say to Jensen's inequality, but don't bet on that and I am not the one to flesh that out even if it is true. $\endgroup$ – Nick Cox Aug 23 '13 at 15:17
  • 2
    $\begingroup$ What causes the "change in value of this expression"? Are you contemplating changing the data, changing the $S_j$, or both? @Nick is correct that there are some inequalities available, such as the Cauchy-Schwarz (apply it to the vectors $(S_j)$ and $(v_j-\bar{v})$), but they won't tell you much: consider the case where all but one of the $v_j$ are a constant $c$ and the last one equals $n\bar{v} - (n-1)c$. $\endgroup$ – whuber Aug 23 '13 at 15:23
  • $\begingroup$ @whuber I meant changes in the data. I'd be interested in hearing more about any inequalities that may apply. I was hoping to find some bound on this expression in terms of $\sigma$. $\endgroup$ – Adam Russell Aug 23 '13 at 15:28
  • 2
    $\begingroup$ Letting $S=(S_j)$ and $v=(v_j)$, your summation (call it $f$) is expressible as a dot product $S\cdot v - \bar{v}\sum S_j$. To learn about changes induced by changing the data, take the partial derivatives $\partial f/\partial v_j = S_j - \bar{S}$ and notice they have nothing to do with $\sigma$. $\endgroup$ – whuber Aug 23 '13 at 15:34
4
$\begingroup$

An inequality

Let $S = (S_0 - \bar{S}, S_1- \bar{S}, \ldots, S_{n-1}- \bar{S})$ and $r = (v_0 - \bar{v}, v_1 - \bar{v}, \ldots, v_{n-1} - \bar{v})$ be the mean-centered $n$-vectors of coefficients and data. Let $\sigma_v = ||r||_2/n$ be the standard deviation of the $v_j$. The Cauchy-Schwarz Inequality (which basically asserts the cosine of the angle between two vectors lies between $-1$ and $1$, reaching those extremes if and only if the vectors are parallel) implies

$$\sum_j S_j(v_j - \bar{v}) = \sum_j (S_j- \bar{S})(v_j - \bar{v}) = S \cdot r \le ||S||_2 ||r||_2 = n ||S||_2 \sigma_v.$$

The first equality exploits the sum-to-zero property of the residuals $\sum_j (v_j-\bar{v}) = 0$, the second is the definition of the dot product, and the last follows from the definition of the standard deviation. The inequality becomes an equality if and only if the vector $r$ is a multiple of the vector $S$.

Assuming all $S_j$ are either $0$ or $1$, this can further be simplified by writing $k$ for the number of nonzero $S_j$, for then $$||S||_2 = \frac{\sqrt{k(n-k)}}{n}.$$

Putting the two results together gives

$$\sum_j S_j(v_j - \bar{v}) \le \sqrt{k(n-k)}\sigma_v.$$

The right hand side is largest when $k=n/2$ or $k=(n-1)/2$ (depending on whether $n$ is even or odd, respectively); in either case the right hand side does not exceed $n/2$ times the standard deviation (and for odd $n$ it is strictly less).

Effects of changing the data

To determine the effect of changing the data $v_0, \ldots, v_{n-1}$ on $f(v_0, \ldots, v_{n-1})$ = $\sum_j S_j(v_j - \bar{v}),$ take the partial derivatives:

$$\frac{\partial f(v_0, \ldots, v_{n-1})}{\partial v_j} = S_j - \frac{1}{n}\sum_j S_j = S_j - \frac{k}{n}.$$

The gradient of $f$ is just the mean-centered vector $S$. This does not involve the data at all, and particularly it's independent of their standard deviation $\sigma_v$.

When the $S_j$ can take other values in the interval $[0,1]$, all the results that do not involve $k$ continue to hold, which includes the two major ones: the implication of the C-S Inequality and the lack of dependence of the gradient of $f$ on $\sigma_v$.

$\endgroup$
4
$\begingroup$

Your original expression can be positive or negative.

Here is an upper bound in terms of the standard deviation, which is greater than or equal to the mean absolute deviation about the mean.

$$\sum_{j=0}^{n-1} S_j(v_{j} - \bar{v}) \le \frac12 \sum_{j=0}^{n-1} \left| v_{j} - \bar{v} \right| = \frac{n}{2} {\displaystyle\sum_{j=0}^{n-1} \left| v_{j} - \bar{v} \right|}{ / n} \le \frac{n}{2} \sqrt{{\displaystyle\sum_{j=0}^{n-1} \left( v_{j} - \bar{v} \right)^2}{ / n}} = \frac{n}{2} \sigma_v$$

To show this is tight, take the example of $v_0 = S_0 = 0$ and $v_1 = S_1=1$.

$\endgroup$
  • $\begingroup$ (+1) But note that your inequality is tight only for even $n$ and can be improved by accounting for the SD of the $S_j$. (For instance, when all the $S_j$ are equal to one another, clearly the sum is $0$.) $\endgroup$ – whuber Aug 23 '13 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.