8

TL;DR you can, but the result would strongly depend on your choice of prior. With maximum likelihood, you would be maximizing the likelihood, that in this case is defined in terms of probability mass function $f$ of Bernoulli distribution, i.e. binomial distribution with number of trials $n=1$, parametrized by probability of success $\theta$ $$ \hat\theta =...


6

You may want t consider the von Mises distribution, aka Tikhonov distribution, and plays the role similar to the normal distribution in 1D statistics: $$ p(\theta ; \alpha, \theta_0 ) = \frac{ e^{\alpha \cos (\theta -\theta_0)}} {2 \pi I_0(\alpha)} $$ For $\alpha=0$ it is uniform, for $\alpha >> 1$ the distribution is sharply peaked at $\theta_0$ C....


5

There are alternatives, for example, you can use constrained optimization, or regularization. Notice however, that in most cases those approaches can be thought as Bayesian inference in disguise. For example, constraining range of the parameter during optimization, is the same as using flat prior over this range. Using $L_2$ regularization is the same as ...


4

The most obvious thing to do here would be to express the variable in polar coordinates and impose a prior on the angle and displacement. That is, you express your point $(x,y)$ as a vector $(\theta,r)$ where: $$\begin{aligned} x &= r \cos \theta, \\[4pt] y &= r \sin \theta. \\[4pt] \end{aligned}$$ You can then impose a prior on $0 \leqslant \...


4

You are confounding a notion from classical statistics ("statistically significant way") with Bayesian statistics, which is, generally speaking, a mistake. In a classical framework, your $P(+|G)$ can't go to zero, instead, it goes to whatever the significance level you are testing at is, e.g., 0.05. At that point, $P(+|G) = P(+|F)$ and cancels out of the ...


4

One way in which prior information can be incorporated into the estimator is through the likelihood (or model, depending on how you look at it). That is to say, when we build a standard parametric model, we are constraining ourselves to say that we are going to allow the model to follow a very specific form, that we know up to the values of the parameters ...


3

You are on the right way. Observe that $$p(\theta_{1:n}, \lambda \vert x_{1:n}) = \prod \limits_{i=1}^n \frac{(m_i\theta_i)^{x_i}}{x_i!} e^{-m_i\theta_i} \prod \limits_{i=1}^n \frac{\lambda^{\alpha}}{\Gamma(\alpha)} \theta_i^{\alpha-1}e^{-\lambda\theta_i} \frac{\sigma^{\beta}}{\Gamma(\beta)} \lambda^{\beta-1} e^{-\sigma \lambda} \\ \propto \prod \limits_{i=...


2

You seem to be confusing some things, so let me clarify. First of all, Bayesian models do not have any special connection to time-series. Same as for non-Bayesian models, using Bayesian framework you can define possibly infinite number of different statistical models. Sure, you can first estimate the model for $n$ time-series points, and then update it for ...


2

I think I understand your confusion. Typically, Bayes' rule is written as: $$p(\theta |y) = \frac{ p(y|\theta) p(\theta)}{p(y)}$$ where $p(\theta |y)$ is the posterior for the observed data $y$ given unknown parameters $\theta$, $p(\theta)$ is the prior distribution, and $p(y)$ is the marginal distribution of $y$. As far as Bayes' rule is concerned, $p(y)...


2

The likelihood function $L(\theta|\mathbf x)$ is defined as a function of $\theta$ indexed by the realisation $x$ of a random variable with density $f(\mathbf x|\theta)$: \begin{align}L\,&:\Theta\longmapsto \mathbb R\\ &\ \ \ \ \theta\longmapsto f(\mathbf x|\theta)\end{align}


2

It's just a normalizing constant which makes the posterior a valid density. In practice, we don't care so much about it. It should be noted that $$p(x) = \int p(x\vert \theta) p(\theta) \, d\theta$$ So it is as if you are averaging the likelihood over the prior.


2

When the number of data values is less than the number of parameters in a statistical model the statistical model does not usually provide anything useful. You have one datum and one parameter (the probability of heads) and the datum does not convey very much information because it is a dichotomous outcome rather than continuous, so don't expect too much. ...


1

I think it'd be easier to calculate the following:$$P(\Theta \leq c| Y = 0)=\int_{0}^c f_{\Theta|Y=0}(\theta)d\theta$$ where $$f_{\Theta|Y=0}(\theta)=\frac{P(Y=0|\Theta=\theta)f_\Theta(\theta)}{P(Y=0)}=\frac{e^{-\theta}\lambda e^{-\lambda\theta}}{P(Y=0)}$$ which is easy to integrate.


1

I agree with @Demetri. Baysian inference is based on cause and effect relationship. $P(effect|cause)$ is usually known. Therefore we try to infer $P(cause|effect)$. Remember effect is observable, hence we try to infer the cause given the effect. $P(data)$ is just a normalizing factor which is total $P(cause)$.


1

If you have a prior for the angle, I'd use it as the reference. E.g. I'd rotate all data so that the prior is at $180^{\circ}$ and measure all angles on the scale $[0^{\circ}, 360^{\circ})$. I see no elegant solution to measuring distance between two angles, $\phi$ and $\psi$. I'd calculate the differences $(\phi - \psi)$ and $(((\phi + 180^{\circ}) \mod ...


1

What do the subscripts of the expectations mean here? They are the distribution you are taking the expectation with respect to. They are the "weights" you're using to calculate the weighted average. I can't really see how... This $$ \mathbb{E}_{p_{\theta}(\mathbf{z})}\left[f\left(\mathbf{z}^{(i)}\right)\right]= \mathbb{E}_{p(\epsilon)}\left[f\left(g_{\...


1

You can check each of your calculations with this table S B C Probability 1 1 1 0.12 1 1 0 0.08 1 0 1 0.03 1 0 0 0.02 0 1 1 0.006666667 0 1 0 0.043333333 0 0 1 0.093333333 0 0 0 0.606666667 This will give you $P(S=1 \mid B=1,C=1) =\dfrac{P(S=1,B=1,C=1)}{P(B=1,C=1)}= \dfrac{0.12}{0.12+0.006666667}\approx ...


1

Consider a binomial model with beta prior $\theta\sim Beta(\alpha,\beta)$ for which we have observed $k$ successes in $n$ attempts. The MLE is $\hat\theta=k/n$, and the posterior mean is $$ E(\theta|y)=\frac{\alpha+k}{\alpha+\beta+n} $$ We see that $E(\theta|x)=\hat\theta$ if $\alpha=\beta=0$. This is known as the Haldane prior $\pi(\theta)\propto \theta^{-1}...


1

The staging does explicitly state anything about the statistical relation of the variables, but rather tells us which conditional distributions we care about. For example, suppose some variable A is a first stage variable and B is a second stage variable. Then, the two probability distributions we care about are $P(A)$ and $P(B | A)$. This is because at the ...


1

In MCMC methods, you typically sample from partially known (as also pointed out in the comments section) distributions. Knowing its form up to a scalar (i.e. constant wrt $\theta$) is enough. Therefore, MCMC can also be used to approximate integrals by implicitly calculating the normalization constant. For example, you probably won't be able to calculate the ...


Only top voted, non community-wiki answers of a minimum length are eligible