# Finding weights that remove residual dot product

I have three time-series $m_i$, $k_i$, $s_i$, for $i \in [1,N]$.

The problem is to find constants $A_m$, $A_k$, $A_s$, such that for $p_i = ( A_m \cdot m_i ) + ( A_k \cdot k_i ) + ( A_s \cdot s_i )$ , and a given integer constant $t$

$\sum_{i=1}^{N-t} { (m_i - p_i) (p_{t+i} - p_i) } = 0$

$\sum_{i=1}^{N-t} { (k_i - p_i) (p_{t+i} - p_i) } = 0$

$\sum_{i=1}^{N-t} { (s_i - p_i) (p_{t+i} - p_i) } = 0$

Not sure if such constants would always exist. Any pointers ?

-
What is time index and what is cross-sectional (I guess it is for each $i$) index in your notations? Or is $i$ corresponding to time? In addition I think you have to choose the objective function to optimize (quadratic loss for instance). –  Dmitrij Celov Aug 2 '11 at 11:55
In requiring an objective function, I guess you mean it is not always possible to find coefficients Am, Ak, As such that the three summations are 0 ? –  Humble Debugger Aug 3 '11 at 19:26

Such constants often exist. We can readily characterize when they do and provide a method to find them.

I presume that nonzero solutions are sought, because $(A_m, A_k, A_s) = (0,0,0)$ always works.

To simplify the notation and reveal what's going on, let $x_1 = A_m$, $x_2 = A_k$, $x_3 = A_s$, and $\mathbf{x} = (x_1,x_2,x_3)'$. Let $U$ be the $3$ by $n$ matrix with rows $\mathbf{m}$, $\mathbf{k}$, and $\mathbf{s}$ and let $U_1$ be the first $N-t$ columns of $U$ and $U_t$ be the last $N-t$ columns of $U$. Set $V = U_t - U_1$. In these terms,

$$\mathbf{p} = \mathbf{x}' U.$$

Similarly, the $t^\text{th}$ difference of $\mathbf{p}$ equals $\mathbf{x}' V$.

Look at the terms in the first equation, for example, and write them in terms of $\mathbf{x}$:

\eqalign{ (m_i - p_i) (p_{t+i} - p_i) &= (u_{1i} - (\mathbf{x}' U_1)_i) (\mathbf{x}' V)_i \\ &= u_{1i}(\mathbf{x}' V)_i - (\mathbf{x}' U_1)_i (\mathbf{x}' V)_i. }

Summing over $i$ and equating to $0$ for the three equations (for $\mathbf{m}$, $\mathbf{k}$, and $\mathbf{s}$) yields the conditions

$$\mathbf{x}' U_1 V' \mathbf{x} = (\mathbf{x}' V U_1')_1 = (\mathbf{x}' V U_1')_2 = (\mathbf{x}' V U_1')_3.$$

Note that the left hand side is a quadratic form in $\mathbf{x}$, $Q(\mathbf{x})$.

Provided $V U_1'$ is nonsingular, we can find an $\mathbf{x}_0$ for which each of the three right hand expressions has the common value of $1$. Assuming $Q(\mathbf{x}_0) \ne 0$, which in general will be the case, set $t = 1/Q(\mathbf{x}_0)$, $\mathbf{x} = t \mathbf{x}_0$, and compute

$$Q(\mathbf{x}) = Q(t \mathbf{x}_0) = t^2 Q(\mathbf{x}_0) = 1/Q(\mathbf{x}_0) = t = t(1) = t(\mathbf{x}_0' V U_1')_i = (\mathbf{x}' V U_1')_i,$$

$i = 1, 2, 3.$ This proves $\mathbf{x}$ is a solution.

We encountered two conditions to assure a nonzero solution: first, $V U_1'$ must be a nonsingular ($3$ by $3$) matrix. (Actually, it suffices that $(1,1,1)$ be in the span of its rowspace.) Second, $\mathbf{x}_0$ must not be in the null space of the quadratic form $Q$ with matrix $U_1 V'$.

### Example

Let the three time series be $\mathbf{m}=(2,4,6,4,2)$, $\mathbf{k}=(2,4,6,8,10)$, and $\mathbf{s}=(8,2,0,2,8)$. Thus

$$U = \left( \begin{array}{ccccc} 2 & 4 & 6 & 4 & 2 \\ 2 & 4 & 6 & 8 & 10 \\ 8 & 2 & 0 & 2 & 8 \end{array} \right).$$

Suppose $t=1$. Then the first differences of the time series (of the columns of $U$, that is) are

$$V = \left( \begin{array}{cccc} 2 & 2 & -2 & -2 \\ 2 & 2 & 2 & 2 \\ -6 & -2 & 2 & 6 \end{array} \right).$$

We compute

$$V'U_1 = \left( \begin{array}{ccc} -8 & -16 & 16 \\ 32 & 40 & 24 \\ 16 & 40 & -40 \end{array} \right).$$

The solution to $\mathbf{x}_0 V U_1' = (1,1,1)$ is

$$\mathbf{x}_0 = \left(-\frac{1}{16},\frac{1}{32},-\frac{1}{32}\right).$$

Then, because $Q(\mathbf{x}_0) = -\frac{1}{16}$,

$$(A_m, A_k, A_s) = \mathbf{x} = (1, -\frac{1}{2}, \frac{1}{2}).$$

As a check, for this solution we have

$$\mathbf{p} = (5,3,3,1,1).$$

Subtracting $\mathbf{p}$ from the rows of $U_1$ gives

$$\left( \begin{array}{cccc} -3 & 1 & 3 & 3 \\ -3 & 1 & 3 & 7 \\ 3 & -1 & -3 & 1 \end{array} \right).$$

The inner products of each of these rows with the first ($t^\text{th}$) differences of $\mathbf{p}$, equal to $(-2,0,-2,0)$, are indeed all $0$, as required.

If you work through the case $t=2$ you will find that this time $VU_1'$ is singular. Nevertheless, there exists a solution $\mathbf{x}_0 = (\frac{1}{32},0,\frac{1}{32})$ to the equation $\mathbf{x}_0 V U_1' = (1,1,1)$, leading to $(A_m, A_k, A_s) = (\frac{1}{2}, 0, \frac{1}{2})$. This shows that nonsingularity of $V U_1'$ is not a necessary condition for a solution to exist: we merely need that $(1,1,1)$ be in the span of its rowspace.

When $t=3$, once more $VU_1'$ is singular, but now there exists no $\mathbf{x}_0$. Thus there cannot be a solution to the original problem in this case. (The three equations are inconsistent.)

-
(+1) the tags for this purely algebraic problem were misleading, resulting in false assumption the $p$ is observed (not latent) response variable, but not the notation for a linear form! I fought if the $p$ is observed, then it is possible to take the $t$ difference for it and transform into linear in unknowns system that has no solution (residuals? regression?). In algebraic case indeed it is exactly as you wrote (I felt I have to do so, but was too sleepy to proceed). Conclusion: I just should not look for econometrics in every post tagged by regression ^_^ –  Dmitrij Celov Aug 4 '11 at 11:16