Return to Question

UPD2 question for @scherm

Although you mention looking at the plot, but I was going for a more automated solution..

I have 22 tracked variables that has >100 values, 57 with >50 values, 192 with >10 values and a bunch more with less values over 5 months period that the data was gathered, 12500 manually logged events/measurements total, 435 variables total(some of them were abandoned of course, but only a small part).

Not a lot of data for statistical analysis but a lot in terms of logging it manually.

Thanks for pointing me towards filling missing data package. Also co-occurence is definitely present. In the end I was looking to build a tool where I'll be able to pick a tracked variable and it will tell me something like picked_event is correlated with event1(n1 steps delay), event2(n2 steps delay), event3(n3 steps delay). But your answer fits my asked question.

I have a question about making a scatter plot thing, I've added a scatter plot of all data points (not interval splitting, each strand is a variable) to my post above, it's sorted by last time tracker was used. I'm not sure what I can gather from it and I'm not sure how would I plot average values for each of the variables as you describe, could you elaborate more on your 3rd point?

So far the only idea that came to me but I haven't tried to implement it is to use a reccurent neural network like LSTM, split each day into 10 minutes pieces, and designate an input for each of the tracked variables and feed them into it, and teach it to predict the next step. After I train it, change/remove some of the variables and see how the overall picture changes. And of course I will have to try and avoid overfitting it since it's just 6 months of data. But I don't think it's the optimal solution. I know nothing about statistics and there should probably be a method to do what I want already. But I'm not even sure how to google for it.

I also intend to add weather data plus I have room temp and CO2 readings as well. So, I have a lot of differently formatted, irregularly logged and not 100% accurate(forgot to log, missed etc) variables tracked in time and I want to find insights about a few of the variables - their causes etc, but I have no idea what influences them and with what delay.

So far the only idea that came to me but I haven't tried to implement it is to use a reccurent neural network like LSTM, split each day into 10 minutes pieces, and designate an input for each of the tracked variables and feed them into it, and teach it to predict the next step. After I train it, change/remove some of the variables and see how the overall picture changes. And of course I will have to try and avoid overfitting it since it's just 6 months of data. But I don't think it's the optimal solution. I know nothing about statistics and there should probably be a method to do what I want already. But I'm not even sure how to google for it.