In a textbook Probability Theory: The Logic of Science written by E. T. Jaynes and others, on page 13 it reads that:
For many years, there has been controversy over ‘frequentist’ versus ‘Bayesian’ methods of inference, in which the writer has been an outspoken partisan on the Bayesian side.
While comparing the structures of the Markov Model and Hidden Markov Model, and discriminating the Markov Chain Monte Carlo (MCMC) approximation from the variational approximation, an idea swims into my mind that the introduction of a hidden layer into Markov Chain is the transition from the frequentist(Markov chain) to Bayesian(Hidden Markov Chain) and MCMC is an frequentist approach but the variational approximation belongs to the Bayesian.
From this answer, if I am not wrong, I conjecture that any task which suits a frequentist approach can be solved also in a Bayesian way, and the vice versa.
Then my question is that if the Bayesian approach is better than the frequentist(1. Jaynes taught me that in that book; 2. for instance more parameters in models like hidden markov model can learn more information and then be better to reach the reality, the posterior distribution than the markov model), then why we still need frequentist methods like MCMC?
In the answers of this question I learned why frequentist is better practically. But I wonder if Bayesian can be as practical as frequentist approaches? How can it be?