We’re rewarding the question askers & reputations are being recalculated! Read more.

New answers tagged

2

In some cases you can. For example, if $X|Y=y \sim N(y,1)$, then the relationship between the characteristic functions of $X$ and $Y$ is $$ \varphi_X(t) = Ee^{itX} = EE(e^{itX}|Y)=E \varphi_{X|Y}(t)=E\varphi^{itY-t^2/2}=e^{-t^2/2}\varphi_Y(t) $$ which you solve for $\varphi_Y(t)$ and then use inversion of the characteristic function to find the distribution ...


0

Ok I figured out a simple example: Let $X_1,X_2$ be iid $N(0,1)$ and independent of $Y$, let $Z = \begin{cases} 1, &\text{if } Y\leq 0\\ 0, &\text{if } Y > 0\end{cases}$, and $X = Z X_1 + (1-Z)X_2$. Then, for any PDF $f_Y(y)$ such that $P(Y\leq 0) = P(Y>0) = \frac{1}{2}$, we have $X\sim N(0,1)$ and also $X|Y=y \sim N(0,1)$.


2

Short answer: Indeed, when the same customer may be approached at most $n$ times, it is optimal to start with offer $y_1=\frac{n-1}{n}x$ and decrease the price by $\frac{x}{n}$ with every refusal. The above result only holds for the uniform distribution of the customer's valuation $v$. Under the normal distribution, closed form answer is feasible only ...


2

First recall that $(\log f)' = \frac{f'}{f}$. We have $$ \frac{\partial}{\partial \theta} \log\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i = \frac{\frac{\partial}{\partial \theta} \int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(b_i;\theta)db_i}{\int p(T_i, \delta_i \mid b_i; \theta)p(y_i \mid b_i;\theta)p(...


2

Exactly! Each class conditional density, i.e. $p_{X|Y}(x|y_k)$ is in Gaussian form, and the overall distribution is the distribution of $X$, i.e. $$p_X(x)=\sum p_{X|Y}(x|y_k)\pi_k$$ ...which assumes that the data X is generated by... $[p_X(x)]$


2

Answer: Posterior of $\sigma^2|Y_1,..., Y_n$ is an instance of inverse gamma distribution with the probability density $$ p(\sigma^2|Y_1,...,Y_n) = \frac{\beta^\alpha}{\Gamma(\alpha)} (\sigma^2)^{-\alpha+1}\exp(-\frac{\beta}{\sigma^2}), $$ where \begin{align} \alpha:=\frac{\nu_0+n}{2}, \quad& \beta:=\frac{\nu_0\sigma_0^2+n(\hat \sigma^2 + \frac{k_0}...


2

There is a saying in computer science that the hardest computer language to learn is your second one. I believe that is true in statistics too. The most challenging probability method to learn is your second. Let’s split these problems apart into a null hypothesis solution and a Bayesian method and, instead, we will try and mentally work through what ...


5

Beta is a conjugate prior for binomial, not the other way around. In beta-binomial model, you assume beta prior for the probability of success $p$. The distribution of $p$ a posteriori follows beta distribution. The posterior predictive distribution of the data follows beta-binomial distribution. If you wanted to compare the Bayesian estimate to the ...


0

Some thoughts I've had: This is similar to wanting to do a two-sample t-test - except that for the second sample I only have a single value, and the 30 values aren't necessarily normally distributed. Correct. The idea is a bit like a t-test with a single value. Since the distribution is not known, and normality with only 30 data points may be a bit ...


2

There's certainly a degree of subjectivity when choosing a priors. Dirichlet distribution is a generalization of beta distribution, so you should probably start with reading about them, and how their parameters influence the distribution. Using the notation introduced in the course $$ \hat P(X = \text{'aardvark'}) = \frac{ \text{# observed 'aardvark'} + \...


4

Let me use the linear regression example, that you mentioned. The simple linear regression model is $$ y_i = \alpha + \beta x_i + \varepsilon_i $$ with noise being independent, normally distributed random variables $\varepsilon_i \sim \mathcal{N}(0, \sigma^2)$. This is equivalent of stating the model in terms of normal likelihood function $$ y_i \sim \...


6

Assumptions in bayesian statistics are generally stronger than those, because you need, in every model, to specify the full distribution of your data and parameters. In many cases, gaussian distribution is used, because of its relation to expected value and arithmetic mean, without really believing in the assumption of normality, and it has been shown that ...


1

It might be helpful to view this chain of events as a binary tree where just two leaf probabilities are relevant. The root node contains all folks who had a 1st interview; we then split this group on being invited to a 2nd interview ("2nd", "no 2nd") and subsequently on whether they felt good about the 1st interview ("good", "bad"). The conditional ...


3

Decision tree with thousands of variables and thousands of nodes would not really be interpretable. Very simple neural network will. There's no clear cut between black-box or not. Moreover it is an optimization algorithm, not an algorithm that works only with black-box functions.


3

It is perfectly fine to do so. One might argue that optimising certain structural hyperparameters like the cost function (e.g. Gini index vs Entropy) is an extreme case of cherry-picking - we pick a model that works for a very specific snapshot of the data with total disregard to the problem mechanics. That assertion is probably correct but that said, ...


4

Stan computes a log-posterior density and uses its gradient to do sampling. It does this by incrementing a variable storing the log probability (really, the log kernel. Ben Goodrich points out that Stan only needs to care about the log probability up to constant terms, which are neglected). At each iteration, each sampling statement in the model block ...


1

You're correct in saying that the classification is not finished when we multiply the conditional probabilities. I guess, the professor is saying "it is just a matter of multiplying ..." because the computationally harder part to do is the multiplication. He just omits mentioning the obvious argmax operation, which is comparably much simple. In several ...


1

Bayes theorem is $$ p(\theta | X) \propto {\overbrace{\vphantom{\prod_{i=1}^N} p(\theta)}^\text{prior}} \; {\overbrace{\prod_{i=1}^N p(X_i | \theta)}^\text{likelihood}} $$ so as the sample size $N$ grows large, the likelihood would play greater role in the posterior. Informally, likelihood enters the formula $N$ times, while the prior only once. Of course ...


0

The problem was in my call to python for $t-distribution$. t_x = t_dist.ppf(q, t_nu, t_mu, t_s) will be t_x = t_dist.ppf(q, t_nu, t_mu, np.sqrt(t_s))


1

I would add a characterization that has not been mentioned so far. A fully Bayesian approach "fully" propagates the uncertainty in all the unknown quantities through the Bayes theorem. On the other hand, Pseudo-Bayes approaches such as empirical Bayes do not propagate all the uncertainties. For example, when estimating posterior predictive quantities, a ...


1

Question 1 : Your model gives you $\mathbf{X}_i = \mathbb{P}(S = s | data[t_i:t_{i+1}])$. I think you want $\mathbb{P}(S = s | data[1:T])$. According to the Bayes rule : $$\mathbb{P}(S = s | data[1:T]) = \mathbb{P}(data[1:T]| S = s)\mathbb{P}(S = s)/\mathbb{P}(data[1:T]).$$ We have that $data[1:T] = \cup_{i=1}^N data[t_i:t_{i+1}]$. We may want to assume ...


3

This kind of problem is an optimisation problem that can either be solved directly from the profit function, or in two-steps using backward induction. To show you how to uses either of these methods, I will first write the optimisation problem out in a helpful mathematical form. I will show the solution by both methods. In this particular case, direct ...


2

Yes, the prior of objective function evaluation in your hyperparameter space is modelled using a Gaussian Process, the target need not be continuous, or have gaussian noise so it seems that it would be appropriate in your case. This assumes some smoothness of your loss function in the hyperparameter space. Similar hyperparameters will have similar ...


1

In this case, it is parameters. Parameters are random in subjective Bayesian thinking and are uncertain in objective Bayesian thinking. For a bivariate normal distribution, you would have five dimensions. You would have both means, the variances and the one covariance which is identical to the other covariance and so does not get separate treatment. If ...


1

I suppose 20 dimensions just means optimization problem for some parameter vector $\theta$ that has 20 coordinates such that $\theta=(\theta_1,...,\theta_{20}) \in \mathbb R^{20}$. They are considering problems of dimension less that 20 so they are solving for less than $20$ values. As the article you refer to say the solution is in $\mathbb R^d$ with $d<...


2

You've written: P(B|A) = P(A|B) * P (B) / P(A) = 1 * 0.8 / 0.5 = 1.6 by which you mean: P(default|smartphone) = P(smartphone|default) * P (default) / P(smartphone) = 1.6 which seems wrong, and indeed is. The problem here is you have forgotten that there is an implicit condition in some of these probabilities, namely that the person actually has a loan (...


0

From the paper: The prior density for the transition matrix is obtained by assuming that the K rows are independent and follow a Dirichlet prior with all hyperparameters equal to two.


4

In non-pathological examples, the prior's variance plays an important role in the "stubbornness" of estimates, especially in hierarchical models. This is very well shown in the 8 schools examples in BDA3. Here is a photo which speaks to this. On the horizontal axis is the prior's standard deviation. As the standard deviation shrinks (alternatively, ...


12

You have an inconsistent set of assumptions. Like saying 80 % of the population of the world like soccer. 100 % of all the people, who like soccer, like also tennis, which implies at least 80 % of the population like tennis. But then you say, 50 % of the population like tennis...! In order to derive $P(A)$, you could specify first $P(A\vert B^c)$ and then ...


23

On the face of it your assumptions are inconsistent in that you think more people will default than have smartphones but you also think all defaulters have smartphones. Part of the problem is that some of your assumptions are for users of your app and part for the whole population and you treat these as being for the same group If instead you just ...


0

Select the optimal decision where classes ={W1,W2} P(x/w1) -->N (2, 0.5 ) normal distribution P(x/w2) -->N (1.5, 2.0) P(w1)=2/3 P(w2)=1/3 Lammda matrix = [ 1 2 ] [ 3 4]


0

For simplicity and to follow the classical state space filtering literature, I will denote by $f(s_t|s_{t-1})$ the transition density of the state $s_t$ given $s_{t-1}$ and $g(z_t|s_t)$ the measurement density of the observation $z_t$ given $s_t$. Independent of your transition and measurement equations, to run the vanilla bootstrap particle filter, you do ...


1

Let's say we have a density $f(x; \theta, \eta)$ and $\theta$ is a parameter of interest but $\eta$ is a nuisance parameter, i.e. we need to know its value to evaluate the density but we don't actually care about it. We want to somehow get rid of $\eta$ and end up with something of the form $g(x; \theta)$. Integration and optimization are two common ...


-1

The tilde is odd. I suspect if the lecturer read the slide, he would skip the tilde and say "the likelihood is binomial." But yes, the idea is that $\mathcal{D}|\theta$ follows a binomial distribution. We talk about the "prior," the distribution of $\theta$ here, and the "likelihood," the distribution of $\mathcal{D}|\theta$ -- this is the distribution of ...


0

Your interpretation of the author's use of $\sim$ seems about right. Based on how the slide is typeset, it seems like saving space was more important than precision. A binomial distribution is characterized by three attributes: a fixed number of binary trials is conducted such that each trial is independent with a fixed probability of success. So it's ...


2

You are on the right track. Substitute expressions for the densities of normals into the Bayes law (and don't keep track of constants !!) \begin{align} f(\theta|X) &\propto \exp(-\frac{1}{2}(X-\theta)^2)\cdot \exp(-\frac{1}{2}\sigma^{-2}\theta^2)\\ & \propto \exp(-\frac{1}{2} ((1+\sigma^{-2})\theta^2 - 2 \theta X + X^2)) \\ & \propto \exp(-\...


0

Meanwhile, other papers related to Bayesian RNNs have been published. For example, Bayesian Recurrent Neural Networks and Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data.


3

Since this is an exercise, I will not solve the whole thing for you, but I will get you started by showing how you would go about finding the conjugate prior kernel. This will give you a basis to then solve the two problems in your question. This first step is essentially just a matter of writing down the likelihood function for your problem, and then ...


1

If $b,c$ are independent you have $p(b,c)=p(b)p(c)$. If $b,c$ are conditionally independent on $a$, you have $p(b,c|a)=p(b|a)p(c|a)$. Also, former doesn't imply the latter in general and vice versa. Also none implies directly what you seek for. Given the information, you should be using the appropriate identity (i.e. the former here) to conclude the ...


2

The answer to (2) is incorrect since $$\theta^{\alpha+x-1}(1-\theta)^{\beta-x}\ne\theta^{\alpha+x-1}-\theta^{\alpha+\beta-1}$$ Similarly, the answer to (3) is incorrect due to the same mistake.


1

Simply looking at generative models on Wikipedia, I read that in machine learning, the generative approach is opposed to the discriminative approach. The generative model corresponds to the conditional density $p(x|Y=y)$ of the data given the model index, while the discriminative model is the conditional probability $\Bbb P(Y=y|X=x)$ of the index given the ...


1

First of all, if you are using Bayesian approach, then every parameters is considered as a random variable, so every effect is "random". The difference between "random" and "fixed" effects is more important in frequentist scenario, where we use different techniques for estimating them. The standard linear regression model could be something like this: $$\...


3

Yes, it does converge to the "true distribution" (suitably defined) First of all, it is worth noting that it is a little strange to refer to the "true distribution" of the parameter as something aside from the prior and posterior. If you proceed under the operational Bayesian approach then the parameter has an operational definition as a function of the ...


2

I have not verified every line of your Question, but it seems your initial formulation for each part is on the right track. In this particular situation, you may be able to save some work by noticing that the beta prior and the binomial likelihood are 'conjugate', as discussed below. [I will not comment on the posterior predictive distribution because you ...


2

Let $p(\theta \mid x)$ be the true posterior and $q_\phi(\theta)$ be the variational distribution (parameterized by $\phi$). The ELBO $\mathcal{L}(\phi)$ can be written as the difference between the log evidence and the KL divergence between the variational distribution and true posterior: $$\mathcal{L}(\phi) = \log p(x) - D_{KL} \Big( q_\phi(\theta) \...


0

The increased complexity of the model is an example of the Neyman-Scott phenomenon. It justifies the need for conditional likelihood to model effects. There are Bayesian and Frequentist analogues of linear mixed models, but none that I know of are amenable to sequential analysis. That doesn't mean you can't store all the data and "update" models by refitting ...


1

First, use a conditional Bayes' rule where we keep conditioning on $X$, $\theta$, and $\mathcal H_i$, and only swap $\mathbf w$ and $\mathbf y$: $$ p(\mathbf w \mid \mathbf y, X, \boldsymbol\theta, \mathcal H_i) = \frac{p(\mathbf y \mid \mathbf w, X, \boldsymbol\theta, \mathcal H_i) \; p(\mathbf w \mid X, \boldsymbol\theta, \mathcal H_i)}{p(\mathbf y \...


0

The answer to this question depends no a lot of stuff but some suggestions are: 1) For modeling the positive real numbers (e.g., revenue) it is usually recommended to use the log-normal distribution (or just take the log of your data and then model it with a normal distribution). As far as prior choice that's an interesting question and you have a lot of ...


6

A divergent transition in Stan tells you that the region of the posterior distribution around that divergent transition is geometrically difficult to explore. For example here is a quote from the manual: The primary cause of divergent transitions in Euclidean HMC (other than bugs in the code) is highly varying posterior curvature, for which small step ...


1

The problem may be that your $g$ depends on $x_i$. Your source says "We now require that $g$ depend on $x_i$ only through the line segment $S_i$." But your $g$ is basically completely flexible; if you were to plot $\lambda$ versus $g(\lambda)$, it could be different with every new point sampled.


Top 50 recent answers are included