Agregating results of a classifier (model) trained online with different parameter values

Question

• Is there any work about training online many instances of the same classifier simultaneously with different parameter settings, in order to have a better classification accuracy ? It is somehow like ensemble methods where however different classifiers are trained instead of the same classifier with different parameter settings (which are trained online).
• What are the challenges that one can consider to resolve in this context ?
• Suppose that we have have a classifier trained with 3 different parameter settings (thus it gives rise to three models M1 M2 M3). If a data-point x is classified as class c1 in model M1, class c1 in model M2 and class c2 in model M3. The most likely class of x here is c1 (maybe), but is there any method that allow us to be more confident about that by computing how likely it is that x is classified as c1 in model M3, and how likely x is classified as c2 in models M1 and M2 (given that c was classified as c1 in M1 and M2, and classified as c2 in M3) ? I don't know if this can help to better classify the data-point x.