3
$\begingroup$

I have a fixed set of outcomes which should typically be determined by a fixed set of inputs (like a multi class problem).Here is a simple workflow:

initial inputs -> suggestion (prediction based on inputs) -> action (performed by humans) -> re-examine current inputs (inputs may change as a consequence of action) -> calculate rating (rating based on same inputs) -> decide if it was the right action or not (goal is to reach maximum rating of 10) -> send feedback to retrain

retraining data comes in once every few days. I'm not sure which kind of approach is best suited for such a problem. If I adopt a classification approach, I will have to create training data myself with sample scenarios ( since each of the independent variables having values between 0 to 10) and then come up with approximate suggestions and guess that if these suggestions are carried out, we get a higher rating.

But this also looks like a state problem to me, where we have the current state of inputs and our suggestions will move them to different value of inputs thereby increasing the overall rating.

I'm not sure of two things here a) Which kind of problem is this? Is it a classification or a MDP problem? b) With such little data coming in at such a large time gap, will either of them (classification / reinforcement) work well?

$\endgroup$
  • $\begingroup$ Is the incoming data affected by the previous incoming data and your previous suggestion? Or is it effectively an independent measure each time that you are trying to guess the best response to? Do you receive any measurement equivalent to a reward (e.g. is the rating something that is returned to you after taking an action)? $\endgroup$ – Neil Slater Oct 6 '17 at 20:09
  • $\begingroup$ New incoming data is not affected by previous incoming data. Rather I'd say new data is a function of previous outcomes. After the user acts on the outcome, it is expected that the rating would increase. The rating is not passed in as an input. It's more like an external reference of the current state. But the rating is calculated using a formula from the same set of inputs. The rating has an upper bound which the system tries to reach. $\endgroup$ – Sujay DSa Oct 7 '17 at 14:44
  • $\begingroup$ Adding the "outcome" is not helping me understand this. I would guess that the "outcome" is determined by the original data, plus the action taken, and some unknown (effectively random) factors? The question is then: Is this, at least in part, a feedback loop? Does the "outcome" have strong influence over changes to next set of inputs? Or does it make more sense, given your understanding of the events you are measuring, to think of each input -> action -> "outcome" as an isolated event where you could optimise separately? Do your inputs capture/summarize any history of previous interactions? $\endgroup$ – Neil Slater Oct 7 '17 at 17:49
  • $\begingroup$ Here's a sample workflow: Initial data -> suggestion -> action -> outcome. If outcome decreases rating: examine input set -> give new suggestion -> update training data with new input/label $\endgroup$ – Sujay DSa Oct 7 '17 at 18:52
  • $\begingroup$ And after outcome? When the next "initial data" arrives, is it affected by the outcome of the previous set? Or are all outcomes separate independent events? The reason I ask this is because that is a key assumption in full reinforcement learning. If it is not the case for you, then perhaps a cut down version will help. Although I would worry that the amount of data you have does make chances of using any kind of ML effectively seem quite low $\endgroup$ – Neil Slater Oct 7 '17 at 18:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.