I have about a million rows of data being collected every day, and I am trying to predict a government figure which is released on a less frequent basis, about once a month. Compared to a traditional ML training set where each example would have an actual value to use for training, I have much smaller training set.
I'm currently just aggregating my daily data to one "monthly" value, and trying to run a regression against the monthly government figure once it's released. Unfortunately, I only started collecting this data, so I don't even have enough data to run a simple regression model.
Is there any better way to do this, that would leverage the millions of rows of data I have? Despite there not being any 'correct answer' for each day of data due to the lower frequency of government data being published?