Machine Usage Forecasting for Hundred Thousands of Machines Across Multiple Locations For work, I am currently working on an algorithm to predict when a machine is used or not. But I got 214k machines in 194k differents rooms in 75k differents locations over 40 countries (that's a lot of data).
Each machine send only when their status change so my raw data look like this :
datetime               |...|location  |room|machineId|status
---------------------------------------------------------
2021-07-15 09:59:56.133|...|building1 |1   |A-1      |InUse
2021-07-15 11:00:05.633|...|building1 |1   |A-1      |Available
2021-07-15 10:59:59.100|...|building1 |2   |B-2      |InUse

Since my boss wants to forecast on a hour-level, I resampled my data as so:
datetime           |...|location  |room|machineId|InUse_cnt
---------------------------------------------------------
2021-07-15 09:00:00|...|building1 |1   |A-1      |1
2021-07-15 10:00:00|...|building1 |1   |A-1      |1
2021-07-15 11:00:00|...|building1 |1   |A-1      |1
2021-07-15 12:00:00|...|building1 |1   |A-1      |0
2021-07-15 10:00:00|...|building1 |2   |B-2      |1

During data exploration tasks, I found out that when a machine is used, InUse_cnt is never higher than 6.
I tried VARMA and Prophet but the results are not good. Then I found out about Croston ([here][1]), I thought this was the good one since it's for intermittent demands but still, the result is very bad (RMSE is around 0.67).
So now, I don't know what to do. I start to think that I did all wrong, maybe I didn't take the problem in the right angle.
Does anyone have worked on similar problems? If so, which methodology would you recommend? Any guidelines would be very appreciated.
I am a junior Data Scientist, this is my first real world problem, before that I only worked on academic projects.
[1]: https://otexts.com/fpp2/counts.html
 A: This is a time series type problem with high cardinality features (i.e. the id of each machine could be a feature in itself, location, city and country etc. are also pretty high cardinality). That suggests to me that neural network based methods with embeddings for these high cardinality features could be attractive (especially if in production use of the model you would be able to re-train after there's a bit of data for each machine so that the model can learn a good embedding for new machines - that helps to address the "cold start problem").
The time series nature of the problem might suggest something like a LSTM model, but even a standard neural network might work.
I would imagine that having local time, flagging typical working hours, flagging public holidays, school vacations, weekend vs. weekday, lagged features that would be available in practice when the model is used (e.g. usage 7 days ago, usage yesterday, usage in last week etc.) etc. as additional features could be helpful, too. Exactly what features are helpful depends of course a bit on what these machines exactly are in what context they are used (and features you cannot get might not be helpful).
The other thing to consider is a good validation strategy to figure out how well the model works when tuning hyperparameters would be a good idea, probably a past-vs-future split (or multiple ones to also cover multiple situations for what the last data may have been - that may also give you a hint how far into the future you can predict well and at what point predictions degrade unless you retrain = clue as to re-training frequency).
