New answers tagged

0

Suppose that we have a sample of $y$ and we are fitting the following model: $$y\sim N(\mu, 1).$$ From this we obtain the posterior of $\mu$. Now we generate $y^{rep}$ from the same distribution: $N(\mu, 1)$. What do we expect when comparing the density of $y$ and $y^{rep}$. If the model is correctly specified, the density of $y$ should be covered by the ...


1

$P(C|A=B_1)=0.2$ because you already know that the brand is $B_1$, and you don't need to multiply with $P(B_1)$ again. What you calculate is actually $P(C\cap \{A=B_1\})$. You've made a similar mistake in the calculation of $P(C)$. It should be $$P(C)=\sum_{i=1}^3 P(C\cap \{A=B_i\})=\sum_{i=1}^3 P(C|A=B_i)P(A=B_i)$$


2

Start with your initial expression for the log marginal likelihood: $$\log p(y|X) = \log \int_{f,u} p(y|f) p(f|u,X,Z) p(u|Z)$$ Multiply by $\frac{q(f,u)}{q(f,u)} = 1$: $$= \log \int \frac{q(f,u)}{q(f,u)}p(y|f) p(f|u,X,Z) p(u|Z)$$ The integral can be seen as an expectation w.r.t. $q(f,u)$: $$= \log E_{q(f,u)} \left[ \frac{p(y|f) p(f|u,X,Z) p(u|Z)}{q(f,u)} \...


1

Sorry that this section proves a challenge! (I corrected the question to indicate that the Uniform on $(a,b)$ is imposing the inequality for all $i$'s, as this is (too) implicit in the book.) There is alas a typo in the book when defining the set (second displayed formula on p.220), not a mistake in your reasoning! The constraint on $(a,b)$ is thus that for ...


1

Have I constructed the Naive Bayes Classifier? Yes, you have. Because, your likelihood calculation was $$P(R,H,N,W'|G)=P(R|G)P(H|G)P(N|G)P(W'|G)$$ If so, what part of these equations is the actual "classifier"? The classifier compares $P(G|R,H,N,W')$ vs $P(G'|R,H,N,W')$ and picks the larger one. This comparison is the classification action. In ...


2

Consider a typical GP regression model with zero mean function, covariance function $k$, and i.i.d. Gaussian noise (with variance $\sigma_n^2$). The posterior predictive distribution $p(y_* \mid X_*, X, y)$ describes the outputs $y_*$ at new test points $X_*$, having seen training data $(X,y)$. It's expectation can be written as: $$\begin{array}{c} E[y_* \...


1

Assume we have $N$ random variables $X_1,…,X_N$ with known (posterior) distributions that are easy to sample from I presume you mean posterior distribution (no "s") , $\pi$ say, since the variables (parameters?) have no reason to be independent a posteriori the expected value of the ten largest of these random variables This means computing $\...


0

I am not sure about $\mathbb{E}_{\theta|D}[X|\theta]$, since the subscript $\theta|D$ implies that the expectation is averaging over all $\theta$, but the $|\theta$ part in the expectation itself implies that $\theta$ has already been observed. Therefore, I will assume that $\mathbb{E}_{\theta|D}[X|\theta] \equiv \mathbb{E}_{\theta|D}[X]$. Using the ...


7

The function you have plotted is the kernel of a beta density function (i.e., it is a positive multiple of the beta density). Since you have really just plotted the binomial likelihood function for a particular observed outcome, from a Bayesian perspective your plotted function is proportionate to the posterior density that emerges from using a uniform ...


2

I can't figure out what PyMC3 does and I find their documentation for Gaussian processes very poorly written. But the idea of Gaussian process (GP) regression is very simple. We have $$ y_i = f(x_i) + \sigma w_i, \quad i =1,\dots,n, $$ and we put a GP prior on $f$, which is a distribution over function $f \sim GP(0, K(\cdot, \cdot))$. I am assuming a zero-...


3

I will assume in this answer that the notion of a multivariate distribution makes sense to you. Given this, let us first consider what a "distribution of functions" means if the functions have finite domain. For example, consider the set of functions $$f: \{1,2,\ldots,n\}\to\mathbb R$$ To specify a function $f$ in this set, we just need to define $...


1

Do you really mean an A/B test (so A and B are the only variations)? I'll assume you do, and answer accordingly. The hierarchical regression modelling approach advocated by Gelman is intended for situations where you have a large number of variations, $A, B, ..., Z$, and wish to compare them pairwise: $P(\lambda_A > \lambda_B), P(\lambda_A > \lambda_C)...


0

The following book might be useful - Title : Bayesian Methodology: An overview with the help of R software. Contents include Bayesian inference, regression, machine learning and Bayesian network (ISBN-13: 978-1092939898)


1

I will try to answer your questions with following order 2,1,3, I believe understanding question 2 will make it easier to understand remaining questions. Question 2: Bayesian inference is done through posterior, not likelihood. The likelihood is just a component of the posterior. Maximizing the likelihood is the frequentist approach since Maximizing the ...


0

I believe the standard way to do predictions when using MCMC is to use each simulated set of parameters to form predictions, then study the distribution of the predictions across all of the simulations. If the model is linear in the parameters then this will give the same results as using the average parameters to form predictions, but in many interesting ...


2

In the event the decision problem "has a value", i.e., when minimax meets maximin, $$\min_\delta\,\max_\theta\,\mathbb E_\theta[L(\theta,\delta(X)]= \max_\pi\min_\delta\mathbb \int \mathbb E_\theta[L(\theta,\delta(X)] \pi(\text{d}\theta)$$ there exists a Bayes procedure or a limit of Bayes procedures that is minimax. (This is actually a ...


5

Joint distribution. Using the graphical model you provided, we get the following joint distribution over all variables of interest, conditioning on model parameters. $$p(\Theta, \mathbf{v} | a_0, b_0, c_0, d_0, \left\{e_0^s, f_0^s \right\}_{s = 0,1}, \left \{ e_0^{s0}, f_0^{s0}, e_0^{s1}, f_0^{s1} \right\}_{s=2:L})$$ In more detail, this gives the ...


0

Depends what you mean by "B,C,D"; if you mean B AND C AND D, that's an intersection (union corresponds to OR). Supposing you mean B AND C AND D, the basic Bayesian formula applies as follows: $$ P(A|(B\cap C\cap D))=\frac{P(A\cap B\cap C\cap D)}{P(B\cap C\cap D)} $$ It is true that this also equals $$ \frac{P(A)P((B\cap C\cap D)|A)}{P(B\cap C\cap D)...


1

You are corrent that Zellner's g-prior depends on the design matrix in the regression. The prior is often used in "empirical Bayesian" analysis where there is no objection to this, but it is not a "pure" Bayesian procedure. One counter-argument to this is that the design matrix is a conditioning object in regression, and so if we treat ...


2

Gaussian-process (GP) regression is almost always handled in a Bayesian manner. When people talk about "picking the right kernel" for their GP, they mean for the prior, and it has a dramatic effect on the predictions they make once updated via data to their posterior. The article you linked makes this idea pretty clear. The author's use of the term ...


2

Under all three hypotheses, $X_1$ and $X_2$ are two uncorrelated random variables with $E[X_1\mid H_i]$ taking on values $0,6,-4$ according as $i=1,2,3$ while $E[X_2\mid H_i]=0$ regardless of the value of $i$, and $\operatorname{var}(X_1\mid H_i) = \operatorname{var}(X_2\mid H_i)=2$ for all choices of $i$. Thus, the likelihoods of the three hypotheses are $$...


1

Using $$p(\theta_1|y) \approx \frac{1}{S}\sum_{s=1}^Sp(\theta_1|\theta_2^{(s)}, y)$$ as an approximation to the marginal posterior density is called a Rao-Blackwellisation in the MCMC literature, started in Gelfand & Smith (1990) foundational¹ paper, appropriately entitled "Sampling-Based Approaches to Calculating Marginal Densities". This ...


1

The following is taken from my manuscript on confidence distributions - Johnson, Geoffrey S. "Decision Making in Drug Development via Confidence Distributions." arXiv preprint arXiv:2005.04721 (2020). In the Bayesian framework the population-level parameter of interest is considered an unrealized or unobservable realization of a random variable ...


1

The following is taken from my manuscript on confidence distributions - Johnson, Geoffrey S. "Decision Making in Drug Development via Confidence Distributions." arXiv preprint arXiv:2005.04721 (2020). In the Bayesian framework the population-level parameter of interest is considered an unrealized or unobservable realization of a random variable ...


0

The following is taken from my manuscript on confidence distributions - Johnson, Geoffrey S. "Decision Making in Drug Development via Confidence Distributions." arXiv preprint arXiv:2005.04721 (2020). In the Bayesian framework the population-level parameter of interest is considered an unrealized or unobservable realization of a random variable ...


0

The following is taken from my manuscript on confidence distributions - Johnson, Geoffrey S. "Decision Making in Drug Development via Confidence Distributions." arXiv preprint arXiv:2005.04721 (2020). In the Bayesian framework the population-level parameter of interest is considered an unrealized or unobservable realization of a random variable ...


1

The following is taken from my manuscript on confidence distributions - Johnson, Geoffrey S. "Decision Making in Drug Development via Confidence Distributions." arXiv preprint arXiv:2005.04721 (2020). In the Bayesian framework the population-level parameter of interest is considered an unrealized or unobservable realization of a random variable ...


0

After skimming some Bayesian Statistics textbooks, I came to the conclusion that the standard Bayesian approach would be to take :$$𝐸[𝜋_𝑛(𝜃)]≥2/3$$ Still, I am not sure about this. Correct. We want the probability of the next toss being heads greater than or equal to $2/3$. This means $$P(H|\mathcal D)=\int P(H|\theta)\pi(\theta|\mathcal D)d\theta=\int \...


1

Think I found the solution: We'll use the chain rule of bayesian networks to calculate the probability of every event being unsuccessful. The answer is 1 minus that number. Is there a better answer out there?


1

I don’t have experience with this package, so for the software specific details you’d need to check the documentation or asks it’s developers, however I can help you with the “posterior mean of a predicted quantity”. Unlike non-Bayesian models, in Bayesian setting we usually find not the point estimates, but their distributions. For example, in frequentist ...


0

I found that through the way of partition the data into independent subsets and compute posterior for each subset, it is then possible to recover the full posterior (Sudipto Banerjee: High-dimensional Bayesian geostatistics).


0

Denote $Y=cX$. The pdf of $Y$ then is $f_Y(y) = \frac{\partial}{\partial y}\int_{-\infty} ^{y/c}f_X(x) dx$ $=\dfrac{1}{\sqrt{2\pi}c\sigma}e^{-\frac{1}{2c^2\sigma^2}(y-c\mu )^2}$


11

A stochastic process $X(t), t \in T$ is a Gaussian process (GP) if $\sum_i a_i X(t_i)$ is a Gaussian random variable for any such linear combination. Equivalently, it is a GP if all its finite-dimensional distributions are (multivariate) Gaussian, that is $(X(t_1),X(t_2),\dots,X(t_n))$ is Gaussian for any choice of $\{t_i\}$. Usually, one in addition ...


3

You are given points sampled from a function, that is not known, this is your data. Traditional curve fitting algorithms would try to find a function that fits the data the best. Gaussian process learns distribution over functions, i.e. you could sample from this distribution the functions that are consistent with the data. This distribution is defined in ...


1

It is not necessary for $p(z)$ and $q(z|x)$ to be normal distributions in a VAE. We can have discrete VAEs too wherein $z$ are discrete. The distribution you've shown for $z$ is completely valid. It's a multinomial/categorical distribution. However, a mistake in your example is that you take $z$ to be observed (you know the value of the sum $S$ = $z$). In ...


3

A mixture of Gaussians$$p_1\mathcal N(\mu_1,\sigma^2_1)+(1-p_1)\mathcal N(\mu_2,\sigma^2_2)$$is only a Gaussian when $$\mu_1=\mu_2\,\quad \sigma_1=\sigma_2$$ or when $$p_1\in\{0,1\}$$


2

The generative story describes how each image sample is generated. The story is as follows - (a) Sample z ~ N(z | 0, I); (b) Sample x ~ N(x | f_mu(z), f_sig(z)) For any generative story, going forward tells us about test time, and going backwards helps us learn the parameters (note that last step has x, our data). Every model comes with its assumptions. The ...


0

Your approach is correct and good, especially if the components are distant from each other. An alternative could be to go one coordinate at a time and think about the conditional probability distribution. Specifically, the CDF of the first parameter is just the weighted sum of the CDFs of the components. Now that you have picked and sampled the first ...


2

Suppose $n=100$ independent observations $X_1, X_2, \dots, X_{100}$ are taken from an exponential distribution and we wish to find a 95% confidence interval (CI) for the population mean $\mu,$ where $E(X_i) = \mu.$ Then one can show that $\frac{\bar X}{\mu} \sim \mathsf{Gamma}(\mathrm{shape}=n, \mathsf{rate}=n).$ Then if $L$ and $U$ cut 2.5%, respectively, ...


1

Is Poisson regression appropriate given data's shape and information supplied? The long answer is maybe but not without some transformations of the data. I took a peak at the data set you referenced and your sales variable is not an integer. The reason the longer answer is a "maybe" is that it is at least conceivable that sales data would count ...


1

I think the argument is the following: We assume the posterior distribution of the weights vary relatively little. eg because priors are sufficiently restrictive or we have a lot of data. A linear taylor series expansion will be effective in a small neighbourhood of a given point. How small depends on how fast the gradient is changing around that ...


2

I looked quickly at the link and didn't see the most direct explanation, but might have skimmed too fast. Expanding "$P(B)$" in this way uses the law of total probability. The idea is that if you have a partition of the sample space, the we can find $P(B)$ by summing up the probability parts of $B$ found in each of the partitions. A partition just ...


0

@ttnphns shared this link that contains the answer: Linear discriminant analysis and Bayes rule: classification Thank you :)


5

Since we're looking for the pdf of $\theta$, we're only concerned with terms that include it. \begin{align} \Pr\left(\theta |\textbf{Z}\right) &\propto \Pr\left(\textbf{Z} \mid \theta\right) \Pr(\theta) \\ &\propto \exp\left(-\frac{1}{2}(z-\theta)^2 -\frac{1}{2\tau}\theta^2 \right) \\ &= \exp\left(-\frac{1}{2}\left((1+\frac{1}{\tau})\theta^2 -2z\...


1

I think you're mixing up How to compute or approximately compute $P(X)$ The "generative story" Also, I might be wrong on this, but based on the last paragraph, it sounds like you're confusing $X$ with the entire dataset, whereas the notation in the article you linked uses $X$ to mean just one point in the dataset. $x_i$ are meant to be the ...


1

Assuming draws w/o replacement based on the answers given, and I'll focus on the second question to form an example for others. Let $B$ be the chosen box ($x$ or $y$), the probability of selecting box $x$ in case we draw two white balls is as follows: $$P(B=x|2W)=\frac{P(2W|B=x)P(B=x)}{P(2W|B=x)P(B=x)+P(2W|B=y)P(B=y)}$$ Here, $P(B=x)=P(B=y)=1/2$, so they ...


0

You are correct in your reasoning. Your full likelihood, with some abuse of notation, is $P(x, \mu, \tau| \mu_0, \alpha, \beta) = \prod_{i=1}^{k}f(x_i|\mu,\tau)P(\mu|\mu_0, \tau)P(\tau|\alpha, \beta)$, so you can get your posterior distributions, \begin{align*} P(\mu|\mu_0, x,\tau) & \propto \prod_{i=1}^{k}f(x_i|\mu,\tau)P(\mu|\tau) \\ P(\tau|x,\alpha, \...


-1

The MVUE estimator (say $f$) achieves the lowest variance out of all estimators such that bias is zero. However, there is no reason why an estimator with non-zero bias (say $f'$) could not have a variance much lower than that of the MVUE, such that overall $\text{MSE}(f) > \text{MSE}(f')$ holds. Consider for example the ridge regression estimator $f_\...


4

First of all the equation is $MSE = Bias^2 + Variance$ and not Bias. Now in MVUE estimators, the Bias is zero and the variance is equal to the CRLB (Cramer-Rao Lower Bound) calculated as follows: CRLB($\hat \theta$) = $\dfrac{1}{-E[\dfrac{\partial^2 \ln P(X;\theta)}{\partial \theta^2}]}$ The MSE then is equal to the variance. To define the Bayes estimators ...


0

We have the following relationship, and you have all the things in the numerator part: $$P(B|C,A)=\frac{P(A|B,C)P(B|C)}{P(A|C)}$$ But, you don't know $P(A|C)$ or anything else that will help you to calculate it.


Top 50 recent answers are included