You seem to mix the notation used by two different sources and the power formulas for one- and two-way repeated measures ANOVAs.

In the (Potvin and Schutz, 2000) paper [1], $n$ is the group sample size (number of subjects in each group) in a balanced design. In the G*Power tutorial [2], $N$ is the total sample size (total number of subjects). The relationship between the group sample size and the total size is simple: $N = pn$ where $p$ in the number of levels of the between-subjects factor A.

Note: (Potvin and Schutz, 2000) denote the number of A levels by $p$; the G*Power tutorial denotes it by $a$ and you denote it by $k$. I'll use the notation in the paper.

On pages 32—36 the G*Power tutorial presents results for a two-way $A_p \times B_q$ repeated measures (RM) ANOVA. You seem to ignore this fact as you are interested in a one-way repeated measures design. To an extent, you sweep the difference between the between-subjects and within-subject factors by referring to "the mean for trial/group $j$", implying that groups and trials are interchangeable.

In your question you reference the formula for the within-subject effect B. Let's show that the G*Power formula is equivalent to the (Potvin and Schutz, 2000) formula.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-\rho} = \frac{npq\sum_{j=1}^q(\mu_j-\mu)^2/(q\sigma^2)}{1-\rho} = \frac{np\sum_{j=1}^q(\mu_j-\mu)^2}{\sigma^2(1-\rho)} = \lambda_B \end{aligned} $$

For completeness, let's also look at the formula for the between-subjects effect A. Aside: In the tutorial there is a typo: the effect size $f^2$ should be in the numerator, not the denominator.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-(1-q)\rho} = \frac{npq\sum_{i=1}^p(\mu_i-\mu)^2/(p\sigma^2)}{1-(1-q)\rho} = \frac{nq\sum_{i=1}^p(\mu_i-\mu)^2}{\sigma^2(1-(1-q)\rho)} = \lambda_A \end{aligned} $$

[1] P. Potvin and R. Schutz. Statistical power for the two-factor repeated measures ANOVA. Behavior Research Methods, 32:347–356, 2012.
[2] G*Power tutorial by StatPower. Available online.

TL;DR The G*Power formula is correct and the (Potvin and Schutz, 2000) formula is also correct. The missing step is how to adapt a formula for two-way ANOVA to the one-way layout.

You seem to mix the notation used by two different sources and the power formulas for one- and two-way repeated measures ANOVAs.

In the (Potvin and Schutz, 2000) paper [1], $n$ is the group sample size (number of subjects in each group) in a balanced design. In the G*Power tutorial [2], $N$ is the total sample size (total number of subjects). The relationship between the group sample size and the total size is simple: $N = pn$ where $p$ in the number of levels of the between-subjects factor A.

Note: (Potvin and Schutz, 2000) denote the number of A levels by $p$; the G*Power tutorial denotes it by $a$ and you denote it by $k$. I'll use the notation in the paper.

On pages 32—36 the G*Power tutorial presents results for a two-way $A_p \times B_q$ repeated measures (RM) ANOVA. You seem to ignore this fact as you are interested in a one-way repeated measures design. To an extent, you sweep the difference between the between-subjects and within-subject factors by referring to "the mean for trial/group $j$", implying that groups and trials are interchangeable.

In your question you reference the formula for the within-subject effect B. Let's show that the G*Power formula is equivalent to the (Potvin and Schutz, 2000) formula.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-\rho} = \frac{npq\sum_{j=1}^q(\mu_j-\mu)^2/(q\sigma^2)}{1-\rho} = \frac{np\sum_{j=1}^q(\mu_j-\mu)^2}{\sigma^2(1-\rho)} = \lambda_B \end{aligned} $$

For completeness, let's also look at the formula for the between-subjects effect A. Aside: In the tutorial there is a typo: the effect size $f^2$ should be in the numerator, not the denominator.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-(1-q)\rho} = \frac{npq\sum_{i=1}^p(\mu_i-\mu)^2/(p\sigma^2)}{1-(1-q)\rho} = \frac{nq\sum_{i=1}^p(\mu_i-\mu)^2}{\sigma^2(1-(1-q)\rho)} = \lambda_A \end{aligned} $$

So can you use G*Power to calculate sample size for a one-way repeated measures ANOVA? Yes: It corresponds to a two-way layout with one level for the A factor, ie, $p=1$. The noncentrality parameter is $\lambda = nqf^2/(1-\rho)$ just as given in your question.

[1] P. Potvin and R. Schutz. Statistical power for the two-factor repeated measures ANOVA. Behavior Research Methods, 32:347–356, 2012.
[2] G*Power tutorial by StatPower. Available online.