Bayesian statistics summarize beliefs whereas frequentist statistics summarize evidence. The Bayesians view probability as a degree of belief. This inclusive and generative type of reasoning is useful for formulating hypotheses. For instance, Bayesians may be able to arbitrarily assign some probability to the notion that the moon is made of green cheese, regardless of whether astronauts have actually been able to travel there to verify this. This hypothesis is perhaps supported by the idea that, from afar, the moon looks like green cheese. Frequentists cannot singularly conceive of a hypothesis that is more than a strawman, nor can they say evidence favors one hypothesis over another. Even maximum likelihood only generates a statistic which is "most consistent with what was observed". Formally, Bayesian statistics allows us to think outside the box and propose defensible ideas from data. But this is strictly hypothesis generating in nature.
Frequentist statistics are best applied to confirm hypotheses. When an experiment is conducted well, frequentist statistics provide an "independent observer" or "empirical" context to the findings by eschewing priors. This is consistent with the Karl Popper philosophy of science. The point of evidence is not to promulgate a certain idea. Plenty of evidence is consistent with incorrect hypotheses. Evidence can merely falsify beliefs.
The influence of priors is generally regarded as a bias in statistical reasoning. As you know, we can make up any great number of reasons for why things happen. Psychologically, many people believe that our observer bias is the result of priors in our brain that keep us from truly weighting what we see. "Hope clouds observation" as the Reverend Mother said in Dune. Popper made this idea rigorous.
This had great historical importance in some of the greatest scientific experiments of our time. For instance, John Snow meticulously collected evidence for the Cholera epidemic and concluded astutely that Cholera is not caused by moral deprivation, and pointed out that the evidence was highly consistent with sewage contamination: note he did not conclude this, Snow's findings predated the discovery of bacteria, and there was no mechanistic or etiologic understanding. A similar discourse is found in Origin of Species. We didn't actually know whether the moon was made of green cheese until astronauts actually landed on the surface and collected samples. At that point, Bayesian posteriors have assigned very, very low probability to any other possibility, and Frequentists at best can say that the samples are highly inconsistent with anything except moon dust.
In summary, Bayesian statistics are amenable to hypothesis generating and frequentist statistics are amenable to hypothesis confirmation. Ensuring that data are collected independently in these endeavors is one of the greatest challenges modern statisticians face.