I am using XGBoost for a classification problem (3 classes) where the 6 features are (unscaled) time series. I tried applying forward chained cross validation, but I obtain very low accuracy with my model, 30-40%.
Here is the code:
from sklearn.model_selection import TimeSeriesSplit from sklearn.metrics import accuracy_score from sklearn.model_selection import cross_val_predict from sklearn.metrics import confusion_matrix from xgboost import XGBClassifier from sklearn.svm import SVC tscv = TimeSeriesSplit() def evaluate(model): # train model on training dataset for train_index, test_index in tscv.split(X2): X_train, X_test = X2[train_index], X2[test_index] y_train, y_test = Y[train_index], Y[test_index] model.fit(X_train, y_train) y_predict = model.predict(X_test) predict_values = [round(value) for value in y_predict] # calculate accuracy accuracy = accuracy_score(y_test, predict_values) print("Accuracy: %.2f%%" % (accuracy * 100.0)) # confusion matrix print("Confusion Matrix:") conf_matrix = confusion_matrix(y_test, y_predict) print(conf_matrix) seed=7 mxgbc = XGBClassifier(max_depth=20, learning_rate=0.1, n_estimators=100, objective='multi:softprob', booster='gbtree', n_jobs=1, nthread=None, gamma=0.1, min_child_weight=4, tree_method='hist',max_delta_step=0, subsample=1, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=1, reg_alpha=0.5, reg_lambda=1, base_score=0.5, random_state=seed,missing=None) print("## XGBClassifier:") evaluate(mxgbc)
The accuracy I get is low, 30-40%. I compared with scaling with MinMaxScaler(), but results are the same. Before moving forward to parameter tuning, I need to know if am I doing something wrong in the model specification/ time series validation above. Help much appreciated.