0
$\begingroup$

I am currently trying to understand the paper "Learning to Predict by the Methods of Temporal Differences" by Sutton. I am stuck with the following step: From "Learning to Predict by the Methods of Temporal Differences" by Sutton, p. 14

(From "Learning to Predict by the Methods of Temporal Differences" by Sutton, p. 14) I do not understand why the equality should still hold after switching the indices k and t and changing the inner sum to go from the beginning to the current temporal difference k instead of running over all temporal differences from t to the end. How is this transformation possible and are there any special assumptions involved for this to hold?

Edit: If I get it right, in essence this is all about proving: $\sum^m_{t=1}\sum^m_{k=t}f(k,t) = \sum^m_{k=1}\sum^k_{t=1}f(k,t)$

I would be grateful for hints on how to accomplish that

$\endgroup$
0
$\begingroup$

Okay, I think I found a solution: As the order of summation does not matter, we can write it in set-form: $\sum^m_{t=1} ... = \sum_{t'} \forall t' \le m$

so

$\sum^m_{t=1}\sum^m_{k=t} f(k,t)=\sum_{t'}\sum_{k'}f(k,t)=\sum_{t',k'}f(k,t), \forall t' \le m, k' \ge t', k' \le m$

and

$\sum^m_{k=1}\sum^k_{t=1}f(k,t) = \sum_{k'}\sum_{t'}f(k,t)=\sum_{k',t'}f(k,t) \forall k' \le m, t' \le k'$

The sets that the sum is built over are equal because $k' \le m, t' \le k' \implies t' \le m$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.