After cross validation and grid search the below are the desired pipeline steps and hyper-params for my model.
k= 19 C= 1.6 model = make_pipeline(SelectKBest(score_func=chi2, k=k), StandardScaler(), LogisticRegression(C=C, solver='lbfgs', max_iter=100)) model.fit(X_train, y_train) predictions = model.predict(X_test)
Is the prediction step correctly applied?
- Should one remove the k best features from X_test prior to prediction ? &
- Should data standard scaler be applied to X_test prior to prediction
I've read plenty of advice regarding not using the test set as part of the learning process of your model and I think the above might be one of the cases. Really looking for best practices.
Thank you in advance