Timeline for Understanding how to batch and feed data into a stateful LSTM
Current License: CC BY-SA 3.0
24 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 2, 2019 at 7:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Dec 2, 2018 at 16:01 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Oct 30, 2018 at 3:00 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Sep 19, 2018 at 13:01 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Aug 9, 2018 at 12:01 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jul 7, 2018 at 2:27 | history | tweeted | twitter.com/StackStats/status/1015422052944285698 | ||
| Jul 6, 2018 at 13:42 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jun 4, 2018 at 16:14 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Apr 7, 2018 at 14:26 | answer | added | T3am5hark | timeline score: 10 | |
| Feb 22, 2018 at 17:21 | comment | added | Aksakal | I would still try overlapping samples with stateless LSTM, but increase the sample size to 50. I would spend more effort on my features and the predicted variable definition. Generally, in order to successfully predict you need to find an invariant. For instance, don't predict the price, but predict return. The price is not a martingale, the return could be. The implication is that for a return you don't need the state, and for a price you do. | |
| Feb 22, 2018 at 17:04 | comment | added | KOB | @Aksakal ok, well maybe Bitcoin prices were a bad example to use. If instead I was using some daily figures that produce a trend where a figure from 50 days ago would still be useful today, how would I then train a stateful LSTM on sequential, non-overlapping batches, and then use the trained LSTM in production daily (and so the daily input would be overlapping - shifted one day forward) | |
| Feb 22, 2018 at 16:59 | comment | added | Aksakal | If you assume that 25 prior day prices have everything you need to forecast tomorrow, then you don't really need to carry the state between samples. Each sample is a sequence, and LSTM going day to day within this sample will calculate the state. Once you jump to a new sample, it forgets the state, unless you tell it to carry it. In economics and finance theory it's taken that the current price holds all the information that is needed to forecast tomorrow. Although it applied to only liquid markets. So, I don't see the reason to carry the states between samples. | |
| Feb 22, 2018 at 16:56 | comment | added | KOB | Why shouldn't I be carrying the state here, and more importantly, what is an example of a problem that a stateful LSTM would be used for? | |
| Feb 22, 2018 at 16:55 | comment | added | KOB | @Aksakal I was just under the impression that this exactly the purpose of the state in an LSTM. The prediction of the price on day 50 could be influences by what the LSTM learnt since day 1, and since I am only feeding in sequences of 25, then carrying the state would allow the LSTM to "remember" information from days 1-25? | |
| Feb 22, 2018 at 16:53 | comment | added | Aksakal | First, why do you want to carry the state? | |
| Feb 22, 2018 at 16:51 | history | edited | KOB | CC BY-SA 3.0 |
added 911 characters in body
|
| Feb 22, 2018 at 16:50 | comment | added | KOB |
That's what I was beginning to think, but then I am failing to understand an example of when a stateful model would be needed? Would it make sense for me to use a stateful LSTM with batches of size 3 (for example), the first of which has the 3 sequences [p1, ... , p25], [p2, ... , p26], [p3, ... , p27], and the next batch would then be the 3 sequences: [p26, ... , p50], [p27, ... , p51], [p28, ... , p52], and so on, with the 3 states per batch being passed on to the next batch?
|
|
| Feb 22, 2018 at 16:46 | history | edited | KOB | CC BY-SA 3.0 |
added 911 characters in body
|
| Feb 22, 2018 at 16:43 | comment | added | Aksakal | I would train stateless with overlapping sequences, then setting the state in prediction is not an issue, just forget the state next day. I wouldn't worry about the state, especially when forecasting prices. | |
| Feb 22, 2018 at 16:39 | comment | added | KOB | No the opposite - I train on non-overlapping, sequential batches, and then make live predictions with overlapping batches (shifted by 1 day for each prediction). I am unsure how I am supposed to handle initialise and then handle the state when using these overlapping batches. | |
| Feb 22, 2018 at 16:28 | comment | added | KOB | @Aksakal I am not actually attempting to predict Bitcoin prices - its just a problem that fits well to what I am trying to explain | |
| Feb 22, 2018 at 16:27 | history | edited | KOB | CC BY-SA 3.0 |
added 98 characters in body
|
| Feb 22, 2018 at 16:23 | comment | added | KOB | "You need to reset the state between what you call batches" - doesn't that defeat the entire purpose of the state? I thought the state is used to store information from previous batches? | |
| Feb 22, 2018 at 16:16 | history | asked | KOB | CC BY-SA 3.0 |