0
$\begingroup$

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type.
I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a conclusion that one predictive model is not enough to get the required efficiency. So I applied a stacking model using Random Forest, K- Nearest Neighbors and Multinomial Naive Bayes algorithms.

Since RF and KNN can take only integer/float values, I was able to encode my 6 categorical features to give as input. But as for the 2 text fields, i had to use NLP libraries(nltk) in python and apply Naive Bayes to get some predictions. However, it has less efficiency. Nonetheless, I took the predictions of all of the above three models and fed it to a third model (again RF or KNN). The prediction outputs on the test set looks good with up to 98% accuracy score.

So, in short, I am sending different features as inputs to the weak learners, and then combining them for input to the stronger model. Is this a valid way of stacking?

enter image description here

$\endgroup$
1
  • $\begingroup$ None of these models are weak learners IMO. Also, applying RF to only 3 inputs seems a bit odd to me. $\endgroup$
    – Scholar
    May 28, 2019 at 14:10

1 Answer 1

0
$\begingroup$

You do NOT want to train your layer 1 models with complete data since that will leak information to layer 2 model.

You should be doing train-test split or k-fold to more accurately test your model. You can check here for more details about how to apply stacking: http://blog.kaggle.com/2016/12/27/a-kagglers-guide-to-model-stacking-in-practice/

$\endgroup$
1
  • $\begingroup$ Thanks for the reply. I have corrected my data leakage problem. But now the accuracy score for test data is 57% while it remains 98% for train data (even with k-folds). The accuracy score is further decreasing for test data in layer 2 model, when it is expected to improve. Do you have any idea why this happening? Why is the model not able to learn? $\endgroup$ May 29, 2019 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.