New answers tagged

1

Yes, it is. As you mentioned, the classical rule is $P(A,B) = P(A|B)P(B)$, but it can also be applied to conditional probabilities like $P(\cdot|C)$ instead of $P(\cdot)$. It then becomes $$ P(A,B|C) = P(A|B,C)P(B|C) $$ (you just add a condition on $C$, but otherwise that's the same formula). You can then apply this formula for $A = y$, $B = \theta$, and $C ...


0

Suppose that $X_{1},\ldots,X_{n}$ are iid exponential random variables, with density function $f(x;\theta)=\theta e^{-\theta x}$. Then the likelihood function will be \begin{equation*} \text{L}(\theta|x)=\prod_{i=1}^{n}f(x_{i};\theta)=\prod_{i=1}^{n}\theta e^{-\theta x_{i}}=\theta^{n} e^{-\theta n\bar{x}} \end{equation*} where $n\bar{x}=\sum_{i=1}^{n}x_{i}.$ ...


1

In his original formulation, Rasch treated ability as fixed --- and such model are nowadays fitted using a conditional approach, which is what pure Rasch modelers prefer, for theoretical reason --- hence the name 1-PL (for item difficulty), but there are other approaches like the joint maximum likelihood technique (poorly recommended) or mixed-effect models, ...


3

Reading verbatim from Dale's History of Inverse Probability he mentions that the first occurrence of the term in English is due to Augustus de Morgan in the 1830's as for instance in his 1834 Encyclopædia Metropolitana but he considers that the first "inverse" perspective was to be in Abraham de Moivre's 1756 edition of his Doctrine of Chances, ...


1

If you are interested in relations between three variables, why are you using two pairwise models? Instead, just build a model that estimates the three means for each of the conditions, and compare them. Technically: build a model that estimates the means, take MCMC samples from the models, and calculate the ratios of interest and their distributions from ...


2

# the collected data ages.16.35 <- c(20,30,60,15,90) ages.36.55 <- c(15,50,30,20,40) ages.older <- c(35,30,10,10, 5) total.16.35 <- 100 total.36.55 <- 60 total.older <- 40 Let the new data be coded as $D=\{v_1, v_2, \ldots, v_5\}$ where $v_i$ is 1 if the video $i$ was liked, 0 otherwise. In the example you give, that is $D=\{0,1,1,0,1\}$...


4

Seeing 0/60 isn't strong evidence for $p\leq 0.01$, since seeing 0/60 is pretty much what you expect for $p\leq 1/60$. This means your posterior won't have high weight on $p\leq 0.01$ unless your prior does If you are confident a priori that $p\leq 0.01$ you want the prior probability of that to be high. A Beta(2,100) prior still only has 26% probability of ...


0

Yes, it's the probability of being greater than 200, since that is when the value is censored. If we let $X$ be the weight, and $Z := X - \theta$ be the standardised normal variable, the probability of not being greater than 200 is $P(X \leq 200) = P(Z \leq 200 -\theta)$. Since $Z$ is a standard normal variable, this is equal to $\Phi(200 - \theta)$, so we ...


0

There are examples in the ABC literature of model selection through Bayes factors. An ecological individual based model example is here: https://doi.org/10.1016/j.ecolmodel.2017.07.017 The paper involves picking between models of different complexity so hopefully it will be useful even if it doesn't deal with SDEs directly. However, I should also mention ...


13

Probability of 'observations' given the 'model' Typically 'probability' is expressed as the probability of an outcome given a particular experiment/model/setup. So the probability is about the frequencies of observations given the model. These types of questions are often not so difficult. For instance, in gambling, we can express the probabilities of ...


2

There are plenty of great answers already, so I'll add a slightly tangential example that I found intriguing. Hopefully its not too far-off from the topic. Markov chain Monte Carlo methods are often used for Bayesian posterior inference. In typical encounters of Markov chains in probability theory, we ask questions like whether a chain converges to some ...


19

"Inverse probability" is a rather old-fashioned way of referring to Bayesian inference; when it's used nowadays it's usually as a nod to history. De Morgan (1838), An Essay on Probabilities, Ch. 3 "On Inverse Probabilities", explains it nicely: In the preceding chapter, we have calculated the chances of an event, knowing the ...


1

It looks to me like $\mathcal{N}(\mu,\tau^2)$ is the prior distribution of $a$. Thus, you don't estimate $\mu$, it is a parameter that describes your initial belief of $a$ -- often this is set to $0$ to encourage low weights unless you have reason to believe another mean is better. So after you see more data, you compute the posterior mean of $a$ using bayes ...


-3

If $p(x)$ is a "normal" probability describing the likelihood or predictability of an event occurring, then inverted probability, $$\frac{1}{p(x)}$$ means the unpredictability, unexpectedness or surprisal that would be associated with that same event if it were to occur. I guess your on the right track by saying that inverse probability lends ...


5

Yes, I believe your thinking is a way to view things in that it points out that the prior is the key ingredient to convert conditional probabilities. My reading is that it is an interpretation of Bayes' theorem, which, as we know, says $$ P(B|A)=\frac{P(A|B)P(B)}{P(A)}. $$ Hence, Bayes' theorem provides the result to convert one conditional probability into ...


2

It's obvious but still someone needs to prove it, i.e. if it doesn't sum up to $1$ for all values of $A$, it's not a distribution (PMF to be more specific) for $A$. Any function which is $\geq0$ and sums up to $1$ for all the values of a RV is qualified to be a distribution for that RV. In this case, $f_{A|B=b}(a)=P(A=a|B=b)$ is nonnegative and sums up to $...


2

Let me first place here your data: pop <- 5e6 brown <- ceiling(.41*pop) green <- ceiling(.42*pop) other <- pop-(brown+green) sample_size <- 7000 brown_in_sample <- ceiling(.38*sample_size) green_in_sample <- ceiling(.44*sample_size) other_in_sample <- sample_size-(brown_in_sample+green_in_sample) My first approach would be to use ...


1

This is tricky. The link function for brms' lognormal is the identity link by default. This means that the underlying Stan model codes the likelihood as mu = Intercept; target += lognormal_lpdf(Y | mu, sigma);, which is leads to an estimate of the median on the log scale. Thus, exp(b_Intercept) should me the median. This is supported by a small example: ...


0

I am stunned at how many introductory books there are. My recommendation is William Bolstad and James Curran's Introduction to Bayesian Statistics. It has several nice elements to it. First, it covers a large percentage of the conjugate problems and covers all the standard problems in a sophomore-level test for an introduction to statistics course. It also ...


2

Your likelihood should have $n$ instead of $n^2$. Anyway, the function you found is proportional to the pdf of a Gamma Distribution.


0

A really introductory book is Will Kurt's Bayesian Statistics the Fun Way. This approaches the logic of Bayesian statistics at a very high level - going through the basics of probability, and how prior information can combine with data to affect posterior probabilities. As an ecologist who didn't have a robust statistical training in gradschool, I still get ...


4

I think I can help. I'll give a partial answer and if you need more then let me know. First I'd like to simplify the notation a bit. Let $X = (x_1, \ldots, x_N)$ denote the data, which is in parallel with $Z = (z_1, \ldots, z_N)$. Also I will suppress the known parameters in the priors for the unknowns $(\pi,Z,\mu,\Sigma)$. With these changes, the joint ...


0

The easiest way would be to recognize that your posterior has the form of an inverse Gamma distribution as $$ h(\sigma^2|y^t) \propto (\sigma^2)^{-t/2-1}\exp\left[-\frac{1}{2\sigma^2}\sum_{i=1}^t y_i^2\right] = \left(\frac{1}{\sigma^2}\right)^{t/2+1}\exp\left[ - \frac{\frac{1}{2}\sum_{i=1}^t y_i^2}{\sigma^2}\right] \propto InvGamma(t/2, \frac{1}{2}\sum_{i=1}^...


0

Having some estimate could be helpful, for example, to determine the approximate number of data points to acquire if a specific uncertainty level of the posterior is desired. This is actually pretty related to the field of "active learning". One key theorem that bounds a "variance" like term is given by Chernoff-Hofdinger bound which ...


0

I can't reproduce this. With cmdstanr... library(cmdstanr) code<-'data { int<lower=0> N; real<lower=0, upper=1> p1[N]; real<lower=0, upper=1> p2[N]; real W1[N]; real W2[N]; real U[N]; } parameters { real<lower=0, upper=1> r; real<lower=0> s; real<lower=0> tau; } transformed parameters { real pred[...


1

The necessary and sufficient condition that the posterior converges to the point mass at the true parameter is that the model is correctly specified and identified, for any prior whose support contains the true parameter. (Convergence here means that, under the law determined by $\theta$, for every neighborhood $U$ of $\theta$, the measure $\mu_n(U)$ of $U$ ...


3

Example: Consider $n = 100$ observations from a population with success probability $\theta = 1/4.$ Suppose the results are as sampled in R below, with $X = 29$ Successes. Frequentist confidence interval. Then a frequentist Agresti-Coull 95% CI for $\theta$ is $(0.210, 0.386).$ set.seed(1015) x = rbinom(1, 100, 1/4); x [1] 29 th.est = (x+2)/(100+4); pm = c(...


0

When they're not equivalent, it could be a case of model misspecification. https://arxiv.org/abs/1806.00550


1

This is a loose notation where $\pi$ means several things related to priors and posteriors: $\pi(p_1,\ldots,p_m,\delta_1|y_1,\ldots,y_m,y)$ denotes the posterior density of the entire collection of (unknown?) parameters $(p_1,\ldots,p_k,\delta_1)$ given the data $(y_1,\ldots,y_m,y)$ $\pi(p_1)\cdots\pi(p_m)$ denotes the prior density of the vector $(p_1,\...


2

Your results look reasonable given your model and your other assumptions. I can't speak to whether the model (and the assumptions) are themselves reasonable. I'm going to change the notation a bit because I like to use the "$p$" to denote a probability density (or mass) function. So I'll use $\theta$ as the probability of "success" ...


0

Frequentist statistics are optimal methods. Maximum likelihood-based statistics are optimal methods. Bayesian statistics are optimal methods. They are each optimal at different things. Each method is very good at solving certain types of problems. For some problems, the differences are minimal enough in practice that the differences are interpretive. ...


1

Statistics, in general, is about learning from the data, that is thought as intrinsically random. There’s no truly deterministic models in statistics. Even when statistician does build a model that is based on some mechanistic model of the phenomenon of interest, she would additionally account for the noise in the data. The name “frequentism” itself is about ...


4

Bayesian inference can be used in any scenario where you could use other forms of statistical inference, e.g. maximum likelihood. Additionally, it has some extra advantages, since it allows you for using priors, so for bringing out-of-data information into the model, and it gives you uncertainty estimates for free, since you learn the distribution of the ...


1

So you are assuming that the coefficient in your model is also random. You can still think of $y_i$ as sum of two normally distributed random variables so it will also be normal. However, the parameters will depend on whether $a$ and $\epsilon_i$ are independent or not. $$E(Y|X, \tau)=xE(a)=\mu x$$ And $$Var(Y|X, \tau) = x^2\tau^2 + \tau^2 + 2xCov(a,\epsilon)...


2

There is no reason for $p(\mathbf x|\mathbf z)$ to factorise in a product of functions of the $z_i$ (for a given $\mathbf x$). The OP seems to ignore the fact that $p(\mathbf x|\mathbf z)$ is a function of $\mathbf z$ as a whole and hence that$$p(\mathbf x|\mathbf z)\prod_{i=1}^m p(z_i)$$has no reason to turn into$$\prod_{i=1}^m \alpha_i p_i(z_i)$$unless the ...


0

It actually doesn't matter, but what will change is "not including 0". For example, a CI not including 0 in an untransformed negative binomial or poisson model, actually doesn't include 0 - meaning the effect is either positive or negative (whatever your values are). Now if you exponentiate (log-link) these values (estimates and CI), then you are ...


1

What you defined in $P(Y|X)=\int P(Y|X,\omega)P(\omega)d\omega$ is the prior predictive function, which is generally used to check if the prior distribution for $\omega$ is reasonable. Notice that the distribution of $\omega$ that appears in this formula is the prior $P(\omega)$, not the posterior $P(\omega|X,Y)$. Now, we can define $x^*, y^*$ as new random ...


1

In the notation of the posterior predictive distribution $$p(y^* \mid x^*, X, Y) = \int p(y^* \mid x^*, \omega)p(\omega, X, Y)\, \text{d}\omega\tag{1}$$ in the question, the posterior density on the parameter vector $\omega$ should be denoted $p(\omega|X,Y)$; $p(\cdot|x^*,x,y)$ is a density function $$p(\cdot|x^*,x,y)\,:\ \mathcal Y \longmapsto \mathbb R^*_+...


11

It's because of your independence assumption, which is not true based on the data. For example, $$P(\text{Outlook=Sunny, Temp=High}|\text{Beach})=1/2$$ because there are 4 situations where you go to Beach and in only two of them the Outlook is Sunny and Temp is High. It's the same situation for the denominator.


3

In Bayesian statistics we condition upon the observed data. The Bayesian part of your statement means that the data are known (and hence fixed to known values) and that the parameters are unknown (and hence allowed to vary and take on any plausible values). In frequentist statistics, on the other hand, we compare the observed data to data that could have ...


5

In frequentist philosophy, parameters are treated as non-random objects, while data are treated as random, hence "parameters are fixed and data vary". In Bayesian philosophy, parameters are treated as random objects, and inference is performed by conditioning on an observed (fixed) set of data, hence "parameters vary and data are fixed". ...


6

Consider the two possible normal populations as models, $p(\mathcal{M_1}), p(\mathcal{M_2})$, that is, two hypothesis that compete to explain your datum $X$. The prior information tells which is the prior odds, $$\frac{p(\mathcal{M_1})}{p(\mathcal{M_2})} = \frac{\pi_1}{\pi_2}$$ You can apply the standard formula $$\underbrace{ \frac{P({\cal M}_1\mid X)}{P({\...


1

That's indeed correct. As mentioned in the comment, it is usually easier to work with the log-likelihood: $$ \log p(y|a,\tau) = \sum_{i} \log p(y_i|a,\tau) $$ You can then easily compute the derivative of the log likelihood with respect to $a$ and $\tau$ to derive the Maximum Likelihood Estimators of these parameters.


2

As also mentioned in the comments, the two tosses are independent only if we know the probability of head of the coin itself. Otherwise, the probability of head can be inferred from past outcomes (in this case there is only one) and the events are dependent, i.e. $P_{X_1,X_2}(x_1,x_2)\neq P_{X_1}(x_1)P_{X_2}(x_2)$.


11

The concept of the likelihood principle (LP) is that the entire inference should be based on the likelihood function and solely on the likelihood function. Informally, the likelihood function is sufficient for conducting inference, meaning that the sampling model and the sample itself can be ignored once the likelihood function is constructed. The example ...


1

Interesting problem. You have made small mistakes in defining probabilities. For example: To put this in the context of the current problem, let $A$ be the probability of observing a choice in $t$ and $B$ be the probability that..... $A$ and $B$ are events not probabilities. So first lets define events that you are interested in. Most importantly, we need ...


1

You're incorrect when you state that "there is no guarantee that 𝑃(𝐴) < 𝑃(𝐵)". I think the problem boils down to using Bayes' rule, when you really just need conditional probability. When you mention 𝑃(𝐴), it's important to remember that it is the overall probability of being in the state 𝐴 at a given time, t. That is, it's the ...


1

From the course notes for MCMCglmm section 1.2 describing prior distributions. For a single variance component the inverse Wishart takes two scalar parameters, V and nu. The distribution tends to a point mass on V as the degree of belief parameter, nu goes to infinity. The distribution tends to be right skewed when nu is not very large, with a mode of $\...


2

The parameter of the Exponential distribution is an inverse scale parameter: $X\sim \mathcal E(\lambda)$ can be represented as$$X=\epsilon/\lambda\qquad\epsilon\sim\mathcal E(1)$$Therefore if $$X \sim \mathcal E(\lambda_0) \\ \lambda_i | \lambda_{i+1} \sim \mathcal E(\lambda_{i+1}) \quad i=0,1,\dots,d\\ \lambda_{d+1} = 1$$ we can write \begin{align}X &= \...


4

Adding three points to the answer by @SextusEmpiricus: First, Doob's Theorem says that the posterior (under correct model specification) converges to the truth except on a set of parameters $\theta$ with prior probability zero. In a finite-dimensional setting you would typically have a prior that puts some mass everywhere, so that a set with prior ...


Top 50 recent answers are included