I have some time series data for prices that I'm trying to perform linear regression on. However, I feel that what I'm doing is incorrect and was hoping someone could point me in the right direction.
The overall background of what I'm doing is taking sentiment analyzed from Twitter data and using that to capture trends in the Korean and American Bitcoin markets and see if Korean markets are more sensitive to trends and social media sentiment.
My data looks like this:
date mentions likes retweets sentiment Volume Close 2017-05-10 0.23 0.2 0.52 -0.24 0.9 0.12512 2017-05-11 -0.12 0.51 0.67 0.8 0.6 0.12353 2017-05-12 0.83 0.12 -0.12 0.23 -0.9 -0.35235 . . . 2019-01-10 0.123 0.27 0.87 0.12 0.52 0.87890
This is just an example DataFrame that I'm working with, but I believe it captures what I'm doing well enough. There are a total of 608 samples from May 10th, 2017 to January 10th, 2019.
Close actually pertain to the price data. The other columns are for Tweets. I used to have several Tweets per day but I've combined all of them and scaled them to be between (-1, 1).
The target variable that I'm attempting to predict is
What I'm trying to do in this particular question is predict values using past time series data with linear regression. My first benchmark is the RMSE, and my plan was to use various models on the same data to compare how they perform in comparison to that benchmark (e.g. SVM, LSTM NN's, etc.)
If I were to write out the steps of my work in order, it would look something like this:
- Prepare data.
- Predict test value data using RMSE.
- Predict test value data using linear regression and compare.
- Use other models.
I've attempted to use linear regression from Python's
sklearn.linear_model.LinearRegression library. When I initially ran it to fit the training data,
date is a string type and so the program alerted me that it cannot work with string data. So I simply dropped the
date column and just worked with the other values in the training and test set.
After dropping the label I inserted it back into the DataFrame after using
sklearn.linear_model.LinearRegression to make predictions and got the following image:
For comparison, the graph I obtained for RMSE is as follows:
The blue and yellow lines together graph out the historical price of Bitcoin. The green lines are the predicted values that have been trained on the training data (blue line). A simple RMSE measure doesn't predict too well as expected, and linear regression also seems to perform very poorly. The poor performance of linear regression was expected, but I get the feeling that something is fundamentally wrong and I'm wondering if I'm understanding implementing linear regression for time series analysis correctly. The Python Scikit-Learn linear regression model uses a basic regression method without any extra functionality (e.g. moving average) as far as I'm aware.
The main concern that I'm feeling is if this is the correct way to implement linear regression with time series data. In my method I disregarded the
date values when training the model, which is an essential part of time series analysis. Is it possible to use linear regression with time series in this case?
In case anyone's wondering what code I used or if it may help. Code is in Python and uses Scikit-Learn, Pandas, and Matplotlib:
train_label, test_label = train.pop('Close'), test.pop('Close') linreg = LinearRegression() linreg.fit(train_.drop('date', axis=1), train_label) prediction = linreg.predict(test.drop('date', axis=1)) rmse = (np.sqrt(np.mean(np.array(test - prediction) ** 2))) test['Predictions'] = 0 test['Predictions'] = prediction train.insert(loc=11, column='Close', value=train_label) test.insert(loc=11, column='Close', value=test_label) plt.plot(train['Close'], label='Training Set') plt.Figure() plt.plot(test['Close'], label='Test Set') plt.Figure() plt.plot(test['Predictions'], label='Predictions') plt.legend() plt.grid(True) plt.show()