How to find Linear regression independent variables for future dates I am new to the forum and Machine learning world. I have been through multiple linear regression tutorials where the example is stock prices.
The dependent variable is closing price and independent variables are date, opening price.
Always, the data set is split in 70:30 train:test respectively, and when predicting we use the test set.
My question is, when we want to predict for dates that are not in the data set, how to find out what is the opening price, because it is also an unknown value in the future, should there be a regression for it as well.
Please help in understanding this problem.
I apologise if I might not have used all the technical terms.
Thanks
 A: Your features are date (pointless) and open price. Your labels are closing price.
What you're not seeing is that the train and test sets are your full-sample historical data set that is already available to you, separated into two parts. No future data enters the model when it is being constructed, meaning that the test set is a make-believe test set meant to simulate what the future unseen data will look like.
with that understood, we can talk about how machine learning prediction tasks are structured:

*

*Say our full-sample historical data contains $100$ time periods that you arbitrarily split into two parts (percentages of the available historical data)

*Training data is historical: features are date and open price, while the label is closing price from time period $1$ through $70$, for example

*Test data is also historical: features are, again, date and open price, and we pretend that this is unlabeled when forming predictions (no closing price attached, even though we have it); this time for time period $71$ to $100$.

*We use the fitted model from the train set to predict the out-of-sample label based on the test features, and then see how the model performed by measuring the regression/learning error between the predicted labels and actual labels that we pretended to not have, as would be the case when we predict labels for actual unseen future data.

As you can see the historical training set comes as one big chunk, while the test set also comes in one big chunk. Therefore, the test set (better called validation set at the model fitting and trial-prediction stage) is a simulation of what future unseen data might look like. Test data is not future data. And seeing that both the training set and test set already come in chunks, there is no reason to be wondering how to prepare one of the features (open price) as if it has yet to become known. You already have open prices in your test set as a series of $30$ $(100-70)$ observations.
One train-test split is not good for financial time series though. This sort of training will give you a sub-optimal model that will overfit the future unseen data. You will have to look into cross-validation techniques instead to avoid overfitting, such as combinatorial cross-validation with purging and embargo. Do not use traditional K-fold or leave-one-out cross-validation since these are not appropriate for time series
besides investing some time in learning the procedure of machine learning in general, you might want to, actually you really should have to, invest time in coming up with more features that might explain closing prices, not just the open price. You are guaranteed to find out afterwards that this is all futile since it is the change in prices (returns) that really matter in finance, and can better allow for meaningful prediction since prices are non-stationary.
