Think like a bayesian, check like a frequentist: What does that mean? I am looking at some lecture slides on a data science course which can be found here:
https://github.com/cs109/2015/blob/master/Lectures/01-Introduction.pdf
I, unfortunately, cannot see the video for this lecture and at one point on the slide, the presenter has the following text:
Some Key Principles
Think like a Bayesian, check like a Frequentist (reconciliation)
Does anyone know what that actually means? I have a feeling there is a good insight about these two schools of thought to be gathered from this.
 A: Per Cliff AB's comment to the OP, it sounds like they are heading towards an Empirical Bayesian philosophy. There are three main Bayesian schools of thought, and Empirical Bayes estimates priors from data, often with frequentist methods. That doesn't conform exactly to the quote (which implies Bayes up front, frequentist-like concerns afterwards), but we shouldn't overlook Cliff AB's excellent comment.
Also, there was, and may still be, a school of Bayesian thought that you don't have to check anything after a Bayesian procedure. More modern thought would use posterior predictive checks, and perhaps that kind of double-check-your-answers approach is what the quote is referring to.
Also, frequentist philosophy is concerned with procedures rather than inferences from data. So perhaps that is also a clue to the quote's meaning.
A: The main difference between the Bayesian and frequentist schools of statistics arises due to a difference in interpretation of probability. A Bayesian probability is a statement about personal belief that an event will (or has) occurred. A frequentist probability is a statement about the proportion of similar events that occur in the limit as the number of those events increases. 
For me, to "think like a Bayesian" means to update your personal belief as new information arises and to "check [or worry] like a frequentist" means to be concerned with performance of statistical procedures aggregated across the times those procedures are used, e.g. what is the coverage of credible intervals, what is the Type I/II error rates, etc. 
A: In the context of this data science class, my interpretation of "check like a frequentist" is that you evaluate the performance of your prediction function or decision function on held-out validation data.  The advice to "think like a Bayesian" expresses the opinion that a prediction function derived from a Bayesian approach will generally give good results. 
A: Bayesian statistics summarize beliefs whereas frequentist statistics summarize evidence. The Bayesians view probability as a degree of belief. This inclusive and generative type of reasoning is useful for formulating hypotheses. For instance, Bayesians may be able to arbitrarily assign some probability to the notion that the moon is made of green cheese, regardless of whether astronauts have actually been able to travel there to verify this. This hypothesis is perhaps supported by the idea that, from afar, the moon looks like green cheese. Frequentists cannot singularly conceive of a hypothesis that is more than a strawman, nor can they say evidence favors one hypothesis over another. Even maximum likelihood only generates a statistic which is "most consistent with what was observed". Formally, Bayesian statistics allows us to think outside the box and propose defensible ideas from data. But this is strictly hypothesis generating in nature. 
Frequentist statistics are best applied to confirm hypotheses. When an experiment is conducted well, frequentist statistics provide an "independent observer" or "empirical" context to the findings by eschewing priors. This is consistent with the Karl Popper philosophy of science. The point of evidence is not to promulgate a certain idea. Plenty of evidence is consistent with incorrect hypotheses. Evidence can merely falsify beliefs. 
The influence of priors is generally regarded as a bias in statistical reasoning. As you know, we can make up any great number of reasons for why things happen. Psychologically, many people believe that our observer bias is the result of priors in our brain that keep us from truly weighting what we see. "Hope clouds observation" as the Reverend Mother said in Dune. Popper made this idea rigorous.
This had great historical importance in some of the greatest scientific experiments of our time. For instance, John Snow meticulously collected evidence for the Cholera epidemic and concluded astutely that Cholera is not caused by moral deprivation, and pointed out that the evidence was highly consistent with sewage contamination: note he did not conclude this, Snow's findings predated the discovery of bacteria, and there was no mechanistic or etiologic understanding. A similar discourse is found in Origin of Species. We didn't actually know whether the moon was made of green cheese until astronauts actually landed on the surface and collected samples. At that point, Bayesian posteriors have assigned very, very low probability to any other possibility, and Frequentists at best can say that the samples are highly inconsistent with anything except moon dust.
In summary, Bayesian statistics are amenable to hypothesis generating and frequentist statistics are amenable to hypothesis confirmation. Ensuring that data are collected independently in these endeavors is one of the greatest challenges modern statisticians face. 
A: It sounds like "think like a Bayesian, check like a frequentist" refers to one's approach in statistical design and analysis.  As I understand it, Bayesian thinking involves some belief about prior situations (experimentally or statistically), let's say for example that the mean reading scores for 4th-graders is 80 words per minute, and that some intervention might increase this to 90 words per minute. These are beliefs based on prior studies and hypotheses. Frequentist thinking extrapolates the findings (of the intervention) to obtain confidence intervals or other statistics that are based on the theoretical and practical frequency or probability of these results happening again (i.e., how "frequently"). For example the post-intervention reading score might be 91 words per minute with a 95% confidence interval of 85 to 97 words per minute and an associated p-value (probability value) of this being different from the pre-intervention score. So 95% of the time, the new reading scores would be between 85 and 97 words per minute after the intervention.   Therefore "think like a Bayesian"---i.e., theorize, hypothesize, look at previous evidence, and "check like a frequentist"---i.e., how frequently would these experimental results occur, and how likely are they to be due to chance rather than the intervention.   
