The Bayesian framework has a big advantage over frequentist because it does not depend on having a "crystal ball" in terms of knowing the correct distributional assumptions to make. Bayesian methods depend on using what information you have, and knowing how to encode that information into a probability distribution.
Using Bayesian methods is basically using probability theory in its full power. Bayes theorem is nothing but a restatement of the classic product rule of probability theory:
$$p(\theta x|I)=p(\theta|I)p(x|\theta I)=p(x|I)p(\theta|xI)$$
So long as $p(x|I)\neq 0$ (i.e. the prior information didn't say what was observed was impossible) we can divide by it, and arrive at bayes theorm. I have used $I$ to denote the prior information, which is always present - you can't assign a probability distribution without information.
Now, if you think that Bayes theorem is suspect, then logically, you must also think that the product rule is also suspect. You can find a deductive argument here, which derives the product and sum rules, similar to Cox's theorem. A more explicit list of the assumptions required can be found here.
As far as I know, frequentist inference is not based on a set of foundations within a logical framework. Because it uses the Kolmogorov axioms of probability, there does not seem to be any connection between probability theory and statistical inference. There are not any axioms for frequentist inference which lead to a procedure that is to be followed. There are principles and methods (maximum likelihood, confidence intervals, p-values, etc.), and they work well, but they tend to be isolated and specialised to particular problems. I think frequentist methods are best left vague in their foundations, at least in terms of a strict logical framework.
For point $1$, getting the same result is somewhat irrelevant, from the perspective of interpretation. Two procedures may lead to the same result, but this need not mean that they are equivalent. If I was to just guess $\theta$, and happened to guess the maximum likelihood estimate (MLE), this would not mean that my guessing is just as good as MLE.
For point $2$, why should you be worried that people with different information will come to different conclusions? Someone with a phd in mathematics would, and should, come to different conclusions to someone with high school level mathematics. They have different amounts of information - why would we expect them to agree? When you are presented knew information, you tend to change your mind. How much depends on what kind of information it was. Bayes theorem contains this feature, as it should.
Using a uniform prior is often a convenient approximation to make when the likelihood is sharp compared to the prior. It is not worth the effort sometimes, to go through and properly set up a prior. Similarly, don't make the mistake of confusing Bayesian statistics with MCMC. MCMC is just an algorithm for integration, same as guassian quadratre, and in a similar class to the Laplace approximation. It is a bit more useful than quadratre because you can re-use the algorithm's output to do all your integrals (posterior means and variances are integrals), and a bit more general that Laplace because you don't need a big sample, or a well rounded peak in the posterior (Laplace is quicker though).