0
$\begingroup$

A team in my company has implemented a basic model-free Q-Learning agent in relation to inventory control. The problem (in my eyes) is that it only knows its reward once per day based on revenue gain for the actions taken. In my experience, model-free agents often take thousands or millions of episodes to train effectively and so we're looking at many years for the agent to converge.

Is this reasoning valid?

I feel like this is an obvious question, I just want to make sure before I call them out.

Improve this question
$\endgroup$
2
  • $\begingroup$ There are a few different ways your statement about reward could be interpreted. Is there one state, one (complex) action and one resulting state plus reward per day? Are there multiple state/action/next state tuples per day, and reward is sparse (IMO probably the most likely/common model)? Are there multiple state/action/next state/reward tuples per day, and reward is only measured for them at the end of the day? $\endgroup$
    – Neil Slater
    May 18, 2018 at 7:20
  • $\begingroup$ This is an interesting question but without further details probably impossible to answer. $\endgroup$
    – Jan Kukacka
    May 24, 2018 at 9:45

1 Answer 1

Reset to default
2
$\begingroup$

I agree it might be quite challenging to train a Q-learning agent with only one datapoint per day, unless the environment is trivial and the state and action space are both very small.

A simulated environment would probably be the best way to do this in a reasonable amount of time.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.