How to combine outputs from different non-machine learning prediction models? I have a list of cancer mutations and I have run five different bioinformatics tools that predict whether a particular mutation will lead to a more aggressive form of cancer or not (0-neutral;1-aggressive). I also have a ground truth set of labels that can be used to judge the performance of the individual tools. So basically I have five lists of 1s and 0s and I want to learn the optimal weights to combine them in order to get the best ensemble model and compare that with the ground truth. Is there any way to accomplish this task in python?
P.S: I have read about stacking classifiers from python but I can't use them here because I am not using traditional classifiers to perform the classification task.
 A: Another approach is create an ensemble learner where you essentially average or use a weighted-average approach to combine the individual predictions. If you have N models (n=5 in your case), then a quick weighted average (or unweighted if $w_i=1$) like the below can be used to average predictions.
$\text{Ensemble} = \frac{1}{n}\Sigma^n_{i} \hspace{1 mm}w_i(\text{model}_i)$
If the predictions are probabilities, then they're on the same scale, and the intepretations essentially do not change (80% is still 80%). If the predictions are hard decision (yes, no), then you can have a median rule or a majority-vote rule (like Random forests use), to decide your final decision.
I actually did something similar to you in the paper below, where my task was predicting differential gene expression. I had 5 or 6 techniques, and the proposed solution was to create an "ensemble learner" of differential expressed gene predictions. This wasn't combining ML models, but it was combining GLMs, which essentially are just another classifier (like an ML) with a prediction, so the framework can help provide guidance.
Zaim, Samir Rachid, Colleen Kenost, Joanne Berghout, Francesca Vitali, Helen Hao Zhang, and Yves A. Lussier. "Evaluating single-subject study methods for personal transcriptomic interpretations to advance precision medicine." BMC Medical Genomics 12, no. 5 (2019): 96.
Hope this helps.
A: where the variables came from does not really matter, so you treat them as any other predictors and fit a model using all these variables that is predicting your outcome, for example, (regularized) logistic regression.
