k-fold cross validation much better than unseen data this is my first "real" project and I am not understanding a certain behaviour.
My dataset spans from 2017 up to today. What I did is cleaning data, getting rid of missing values etc.
There are mixed features numerical and categorical, I use different pipelines for them and a feature union to bring it all together, so far so good.
I looked further into 2019 data. We are talking of ~400k observations. Target Variable is binary and I approached this with RandomForests first.
20% of the data is Test data, I generate this with train_test_split of sklearn. Here it starts to get a bit strange to me. 5-fold cross-validation on the 80% train data gets me quite good results, the score I chose for now is F1 and its always around 94%. 
Also the Test data is around 92-94% F1 score. 
However, when I manually let a model train with Jan 2019 to October 2019, the Recall for predicting the Rest of 2019 is going down to 60-70%.
It gets even worse when I train the model with all of 2019 data (or the 80% mentioned earlier) and give it a shot for 2020. I get a very very low recall. I have to say that only a small fraction of the data hits the target variable so it's basically a small group we are trying to find. The precision is quite high as is the accuracy but recall is very jumpy.
My main question is why does the 5-fold cross-validation on 2019 data is bringing F1 and recall > 90% and when I select 10 out of 12 months for training and try to predict the other 2 it is so much worse.
I thought of overfitting but shouldn't the 5-fold cross-validation have picked this up then?
Could anybody point me in the right direction?
edit: I seem to have broken it down in a hopefuly basic issue.
So here is what I don't get.
I have this static set of data of 2019 now.
When I split it with 
X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.2)
finalpipe.fit(X_train,y_train.values)
predic = finalpipe.predict(X_test)

I get a very high score no matter what I am looking at. Accuracy, Recall, Precision all >78%
Now I am doing cross_val_score on it like this 
scores = cross_val_score(finalpipe, features, labels.values.ravel(),cv=5)

and the score is terrible not close to 78% at all, more like 50-60%
I am aware the split of train_test_split and cv is random, but I repeated this multiple times and no matter how the data looks like I should come to the same results, shouldn't I ? Its both splitting data in 80/20 only that cross_val_score is doing it 5 times and averages but again, when I am looping the first one and average myself its still not anywhere close to what cross_val_score gets me
 A: I am answering your questions based on the following assumption:
1: For k-fold cross validation you are only using 2019 data.
Following may be the reason for your mentioned problem:
1: The dataset which you are using of only 10 months may be skewed means one class will be in higher majority than other. But in k-fold, it is randomly suffling the data of one year and then from that data splitting the train and validation set. May be the resulting train set is not that much skewed. 
If your dataset is skewed then model prediction is more biased towards majority class.
2: In k-fold, split is happening to make sure that ratio of two classs will be same in both train and validation set but this is not the case for your manual testing data. 
3: There may be data leakage in the k-fold split but there is not data leakage in your manual train and test split.
4: Data of 10 months may be very less than 80% of the whole data, and rest of the two month test data is very large. This is why it is not properly getting trained.
.
