Can I use the probability score generated from a Machine Learning model as a feature in another model? For example, say we have a model which generates the probability of an ad being bad. Lets call it badness_score. I am working on building another model which will predict the probability of an advertiser being a fraudster. Can I use badness_score as a feature while building this model ? Are there any caveats to this approach?
It's perfectly OK, and also widely used. Different models can explain different perspectives of data and stacking them in front of each other, and using outputs/predictions produced by previous layers enables even moderately simple final layer algorithms to perform much better compared to on their own, because they use the cumulative knowledge learned via other algorithms. This is somewhat analogous to adding layers to neural networks.