As far as I studied, the Bayesian approach is the most correct in Machine Learning. Thought, I solved an exercise where it was required to find out the decision boundary in predicting two different classes, assumed to be generated with a 2D Gaussian distibution.
Here is is the Bayesian approach. The boundary is: $$C_1 \quad if \quad P(x|C_1)P(C_1) > P(x|C_2)P(C_2)$$ $$C_2 \quad else,$$ where $P(x | C_1)$ and $P(x | C_2)$ are assumed to be Gaussian (the parameter are found with max-log likelihood), $P(C_1) = N_1 / (N_1 + N_2)$ and $P(C_2) = N_2 / (N_1 + N_2)$.
Now the problem is that, graphically speaking, the frequentist approach seems to be more precise, while the bayesian one seems to exaggerate the blue area which clearly cover an area that is dominated by red samples. How will you justify the usage of bayesian approach?