As a part of my Master Thesis I plan to use news headlines as an input feature for a time series model that predicts the daily trend of Bitcoin returns (1= positive return, 0= negative return). Besides the headlines, I also ulilize other Macro economic variables possibly related to Bitcoin, which come as integers in a daily frequency (e.g. interest rates, etc.).
So far I scraped around 16,000 news headlines over the past 5 years (8-10 headlines per day). Each headline consists of max 30 words. The headlines are cleaned (lower case, punctuation and number removal, stop word removal and lemmatized) and tokenized on a word level.
The first step in order to use the headlines as an input is to represent the raw text as vectors. After quite some research I chose word embedding algorithms to do the job. I am working in R and tried a couple approaches of implementing word2vec or GloVe in order to train the word vectors. Finally, I trained the word vectors using the GloVe algorithm implemented in the “text2vec” R package.
Assuming, I now have 50 dimensional word vectors for all words in the news headlines.
- How do I retrieve the original structure of the news headlines and bring the word vectors in a form that is compatible with the other input features (Macro variables)?
My thought were: The resulting data frame should look like a classic time series or xts object (date in rows, features in columns). Since I have max 10 headlines per date, every headline has max 30 words and every word is a 50-dimensional vector, the news headlines for one date have to be represented through 10*30*50=15,000 columns.
- Is that reasoning correct?
In case it is: After deleting all 'only zero' columns the dimensionality shrunk down to 8,700 columns which is still way too high. A PCA (90% of the explained variance is contained) reduced the dimensionality to around 1,000 columns. However, my initial goal was to have even a lower dimensional representation of the news headlines per date.
- How do I achieve this without losing too much information? Might doc2vec be an option instead of word embeddings like word2vec or GloVe?
Your help and advice is much appreciated. In case you have further questions or need more details/context, just let me know.
Thanks a lot in advance and I am looking forward to your answers :)
*Edit: I deleted redundant parts of the question and narrowed it down to be more specific. I hope it is specific enough so that the question can be opened again.