251

Let me first explain what a conjugate prior is. I will then explain the Bayesian analyses using your specific example. Bayesian statistics involve the following steps: Define the prior distribution that incorporates your subjective beliefs about a parameter (in your example the parameter of interest is the proportion of left-handers). The prior can be "...


159

Say 200 people took the interview, so that 100 received a 2nd interview and 100 did not. Out of the first lot, 95 felt they had a great first interview. Out of the 2nd lot, 75 felt they had a great first interview. So in total 95 + 75 people felt they had a great first interview. Of those 95 + 75 = 170 people, only 95 actually got a 2nd interview. Thus the ...


108

I will focus this answer on the specific question of what are the alternatives to $p$-values. There are 21 discussion papers published along with the ASA statement (as Supplemental Materials): by Naomi Altman, Douglas Altman, Daniel J. Benjamin, Yoav Benjamini, Jim Berger, Don Berry, John Carlin, George Cobb, Andrew Gelman, Steve Goodman, Sander Greenland, ...


107

Simple linear regression model $$ y_i = \alpha + \beta x_i + \varepsilon $$ can be written in terms of probabilistic model behind it $$ \mu_i = \alpha + \beta x_i \\ y_i \sim \mathcal{N}(\mu_i, \sigma) $$ i.e. dependent variable $Y$ follows normal distribution parametrized by mean $\mu_i$, that is a linear function of $X$ parametrized by $\alpha,\beta$, ...


107

Well by distributing the $P(H)$ term, we obtain $$ P(H|X) = \frac{P(X|H)P(H)}{P(X)} P(C) + P(H) [1 - P(C)], $$ which we can interpret as the Law of Total Probability applied to the event $C =$ "you are using Bayesian statistics correctly." So if you are using Bayesian statistics correctly, then you recover Bayes' law (the left fraction above) and if you aren'...


103

Let $\text{pass}=$ being invited to a second interview, $\text{fail}=$ not being so invited, $\text{good}=$ feel good about first interview, and $\text{bad}=$ don't feel good about first interview. $$ \begin{align} p(\text{pass}) &= 0.5 \\ p(\text{good}\mid\text{pass}) &= 0.95 \\ p(\text{good}\mid\text{fail}) &= 0.75 \\ p(\text{pass}\mid\text{...


80

The Dirichlet distribution is a multivariate probability distribution that describes $k\ge2$ variables $X_1,\dots,X_k$, such that each $x_i \in (0,1)$ and $\sum_{i=1}^N x_i = 1$, that is parametrized by a vector of positive-valued parameters $\boldsymbol{\alpha} = (\alpha_1,\dots,\alpha_k)$. The parameters do not have to be integers, they only need to be ...


73

It is a very broad question and my answer here only begins to scratch the surface a bit. I will use the Bayes's rule to explain the concepts. Let’s assume that a set of probability distribution parameters, $\theta$, best explains the dataset $D$. We may wish to estimate the parameters $\theta$ with the help of the Bayes’ Rule: $$p(\theta|D)=\frac{p(D|\...


67

Stats is not Math First, I steal @whuber's words from a comment in Stats is not maths? (applied in a different context, so I'm stealing words, not citing): If you were to replace "statistics" by "chemistry," "economics," "engineering," or any other field that employs mathematics (such as home economics), it appears none of your argument would change. ...


61

The Dirichlet distribution is a conjugate prior for the multinomial distribution. This means that if the prior distribution of the multinomial parameters is Dirichlet then the posterior distribution is also a Dirichlet distribution (with parameters different from those of the prior). The benefit of this is that (a) the posterior distribution is easy to ...


60

Here's five reasons why frequentists methods may be preferred: Faster. Given that Bayesian statistics often give nearly identical answers to frequentist answers (and when they don't, it's not 100% clear that Bayesian is always the way to go), the fact that frequentist statistics can be obtained often several orders of magnitude faster is a strong argument. ...


58

I'm going to take your questions in order: The question is, Who are the Bayesians today? Anybody who does Bayesian data analysis and self-identifies as "Bayesian". Just like a programmer is someone who programs and self-identifies as a "programmer". A slight difference is that for historical reasons Bayesian has ideological connotations, because of the ...


51

The general term Naive Bayes refers the the strong independence assumptions in the model, rather than the particular distribution of each feature. A Naive Bayes model assumes that each of the features it uses are conditionally independent of one another given some class. More formally, if I want to calculate the probability of observing features $f_1$ ...


49

This is a good question, that seems to pop up a lot: link 1, link 2. The paper Bayesian Estimation Superseeds the T-Test that Cam.Davidson.Pilon pointed out is an excellent resource on this subject. It is also very recent, published in 2012, which I think in part is due to the current interest in the area. I will try to summarize a mathematical explanation ...


44

Statistical Rethinking, has been released just a few weeks ago and hence I am still reading it, but I think is a very nice and fresh addition to the really introductory books about Bayesian Statistics. The author uses a similar approach as the one used by John Kruschke in his puppy books; very verbose, detailed explanations, nice pedagogical examples, he ...


44

This has been discussed in my paper (published only on the internet) "On an Example of Larry Wasserman" [1] and in a blog exchange between me, Wasserman, Robins, and some other commenters on Wasserman's blog: [2] The short answer is that Wasserman (and Robins) generate paradoxes by suggesting that priors in high dimensional spaces "must" have ...


44

I do not see much appeal in this example, esp. as a potential criticism of Bayesians and likelihood-wallahs.... The constant $c$ is known, being equal to $$ 1\big/ \int_\mathcal{X} g(x) \text{d}x $$ If $c$ is the only "unknown" in the picture, given a sample $x_1,\ldots,x_n$, then there is no statistical issue about the problem and I do not agree that there ...


44

The main issue is that the first experiment (Sun gone nova) is not repeatable, which makes it highly unsuitable for frequentist methodology that interprets probability as estimate of how frequent an event is giving that we can repeat the experiment many times. In contrast, bayesian probability is interpreted as our degree of belief giving all available prior ...


44

IMHO "yes"! Here is one of my favorite quotes by Greenland (2006: 767): It is often said (incorrectly) that ‘parameters are treated as fixed by the frequentist but as random by the Bayesian’. For frequentists and Bayesians alike, the value of a parameter may have been fixed from the start or may have been generated from a physically random ...


44

I will consider both Matloff's points: With large samples, significance tests pounce on tiny, unimportant departures from the null hypothesis. The logic here is that if somebody reports highly significant $p=0.0001$, then from this number alone we cannot say if the effect is large and important or irrelevantly tiny (as can happen with large $n$). I find ...


43

Suppose that you and a friend are analyzing the same set of data using a normal model. You adopt the usual parameterization of the normal model using the mean and the variance as parameters, but your friend prefers to parameterize the normal model with the coefficient of variation and the precision as parameters (which is perfectly "legal"). If both of you ...


42

Because, assuming normal errors is effectively the same as assuming that large errors do not occur! The normal distribution has so light tails, that errors outside $\pm 3$ standard deviations have very low probability, errors outside of $\pm 6$ standard deviations are effectively impossible. In practice, that assumption is seldom true. When analyzing ...


42

The (frequentist) bootstrap takes the data as a reasonable approximation to the unknown population distribution. Therefore, the sampling distribution of a statistic (a function of the data) can be approximated by repeatedly resampling the observations with replacement and computing the statistic for each sample. Let $y = (y_1,\ldots,y_n)$ denote the ...


42

Even though variational autoencoders (VAEs) are easy to implement and train, explaining them is not simple at all, because they blend concepts from Deep Learning and Variational Bayes, and the Deep Learning and Probabilistic Modeling communities use different terms for the same concepts. Thus when explaining VAEs you risk either concentrating on the ...


41

I agree that the example is weird. I meant it to be more of a puzzle really. (The example is actually due to Ed George.) It does raise the question of what it means for something to be "known". Christian says that $c$ is known. But, at least from the purely subjective probability point of view, you don't know it just because it can in principle be known. (...


41

Posterior predictive checks are, in simple words, "simulating replicated data under the fitted model and then comparing these to the observed data" (Gelman and Hill, 2007, p. 158). So, you use posterior predictive to "look for systematic discrepancies between real and simulated data" (Gelman et al. 2004, p. 169). The argument about "using the data twice" is ...


41

I believe that you and your colleague are correct. Statistics.com has the correct line of thinking, but makes a simple mistake. Out of the 90 "OK" claims, we expect 20% of them to be incorrectly classified as fraud, not 80%. 20% of 90 is 18, leading to 9 correctly identified claims and 18 incorrect claims, with a ratio of 1/3, exactly what Bayes' rule yields....


39

You ask about three things: (a) how to combine several forecasts to get single forecast, (b) if Bayesian approach can be used in here, and (c) how to deal with zero-probabilities. Combining forecasts, is a common practice. If you have several forecasts than if you take average of those forecasts the resulting combined forecast should be better in terms of ...


39

A half-Cauchy is one of the symmetric halves of the Cauchy distribution (if unspecified, it is the right half that's intended): Since the area of the right half of a Cauchy is $\frac12$ the density must then be doubled. Hence the 2 in your pdf (though it's missing a $\frac{1}{\pi}$ as whuber noted in comments). The half-Cauchy has many properties; some are ...


38

First of all, there is no such a thing as uninformative prior. Below you can see posterior distributions resulting from five different "uninformative" priors (described below the plot) given different data. As you can clearly see, the choice of "uninformative" priors affected the posterior distribution, especially in cases where the data itself did not ...


Only top voted, non community-wiki answers of a minimum length are eligible