0
$\begingroup$

I have an objective to minimize the transmission delay (D) and energy cost (E) for a wireless network device.

While I am solving it using reinforcement learning (Q-Learning to be exact), hence I have to solve it to find the maximum for the value function. Hence (simplistically) if my Cost is $C(t) = T(t) + D(t)$ and I have to find $\min \gamma ^t C(t)$, is this mathematically equivalent to the following?

$$\max \gamma^t \frac{1}{C(t)}$$

Does it means if I solve for the above expressions, I practically solved the $\min \gamma^tC(t)$ problem?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

If your cost is strictly positive, then yes, that will work - theoretically. Your optimizer may have numerical problems with any derivatives because of the reciprocal.

It's usually far easier to just maximize the negative: $-\gamma^tC(t)\to\max$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ On this point, can you please guide me between the difference in using additive versus multiplicative inverse for transforming a maximization problem into a minimization problem (and vice versa)? One that I can see from your answer is the problem with finding derivate. $\endgroup$
    – SJa
    Commented Aug 5, 2020 at 9:34
  • 2
    $\begingroup$ Well, the main advantage is that if you have the derivative $\frac{d}{dt}C(t)$, then the derivative of $-C(t)$ is extremely easy to calculate, it's just $-\frac{d}{dt}C(t)$ - simple, fast, easy, numerically exactly as stable as the original derivative. The derivative of $\frac{1}{C(t)}$ is more problematic: $\frac{d}{dt}\frac{1}{C(t)}=-\frac{1}{\big(\frac{d}{dt}C(t)\big)^2}$. Zeros in $\frac{d}{dt}C(t)$ turn into divisions by zero in the quotient. You really don't want that for numerical reasons. $\endgroup$
    – Stephan Kolassa
    Commented Aug 5, 2020 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.