Questions tagged [bayesian]
Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.
2,721
questions with no upvoted or accepted answers
13
votes
0
answers
708
views
Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)
Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as
$$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$
Here the kernel $K$ is an assigned smooth ...
- 140k
12
votes
0
answers
447
views
Official name of a common type of Bayesian simulation study
There is a kind of simulation study that is commonly used to validate an implementation of a Bayesian model:
For independent replication $i = 1, ..., n$:
Draw a set of "true" parameters ...
- 267
11
votes
2
answers
699
views
Hypergeometric: how do I construct a credibility interval around K (population successes) in R?
I have a problem for which I believe I should use the hypergeometric distribution, but I can't figure out how to do it in R.
Say I have a bag of marbles with known number ($N$) of marbles, but the ...
- 211
10
votes
3
answers
191
views
How to guess the size of a set?
Assume we have a set of unique words and draw a number $n$ of them using simple-random-sampling without replacement independently in each round. We have several rounds and try to guess the set size ...
- 203
10
votes
0
answers
179
views
Pope effect on pizza - Regression with presence absence and similarity data as dependent variables
I'm trying to figure out the right way to set up a regression when the dependent variables are presence absence data (of pizzas), and the similarity between the present pizzas. Bear with the story:
...
- 241
10
votes
0
answers
3k
views
Fourier transform of a Gaussian process
I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense.
For that purpose, let me describe the following situation.
Let $z(s)$ be a ...
- 101
10
votes
0
answers
147
views
Rationale behind Good–Turing frequency estimation?
Good–Turing frequency estimation is a smoothing estimator for estimating a multinomial distribution. It seems very convoluted.
From mathematical statistics point of view, what is the rationale
...
- 19.7k
9
votes
1
answer
2k
views
PyMC3 implementation of Bayesian MMM: poor posterior inference
Google released a whitepaper on Media Mix Modelling (MMM) in 2017; vanilla MMM (established in the 1960s) uses multivariate regression. It's a decent mechanism to understand which of your marketing ...
- 3,398
9
votes
0
answers
11k
views
Singular fit with simplest random structure in lmer (lme4), is a Bayesian approach the only option?
I'm running a mixed model with the lmer function from the lme4 package in R and ran into some issues with singular fits. I get the warning message 'singular fit', ...
- 91
9
votes
0
answers
3k
views
Implementing Predictive Posterior Distribution Using Stan
Background
I had an example that sought to demonstrate the posterior predictive distribution in the context of a normal measurement model. The data that was used is as follows:
...
- 2,144
9
votes
0
answers
3k
views
Mean, median, or mode of skewed posterior?
I'm estimating an ICC from 2 and 3-level hierarchical models using rstanarm. The simplest models are: y ~ (1|group) or ...
- 435
9
votes
0
answers
1k
views
Density estimation/approximation from MCMC samples
I'm looking to accurately describe the density function of a multivariate posterior probability distribution based on samples from MCMC. As far as I know, in most cases this is done either with a ...
- 91
9
votes
0
answers
204
views
Generalization of degrees of freedom for t distribution for coefficients after multiple imputation
Donald Rubin has shown that regression coefficient estimates have fatter tails after multiple imputation and has provided a formula for the degrees of freedom to use as a t-distribution approximation ...
- 96.5k
9
votes
0
answers
3k
views
Horseshoe priors and random slope/intercept regressions
I'm interested in using the horseshoe prior (or the related hierarchical-shrinkage family of priors) for regression coefficients of a traditional multilevel regression (e.g., random slopes/intercepts)....
- 701
8
votes
0
answers
62
views
Priors as Controls : Bayesian Regression
I have a general question about Bayesian Regression Modeling and how a prior might be used as a means to control for (close to) simultaneous events. I often face a situation where I have a time series ...
- 8,780
8
votes
0
answers
388
views
Why does Quadratic (Normal/Laplace) Approximation fail on multilevel models?
In Statistical Rethinking, 2nd Edition, section 13.1, Richard McElreath says:
Why doesn’t simple quadratic approximation, using for example quap, work with multilevel models? When a prior is itself a ...
- 101
8
votes
0
answers
109
views
Trouble replicating simple example of Bayesian inference
On pages 20-21 of John Kruschke's Doing Bayesian Data Analysis book (2nd ed.), there is an introductory illustration of Bayesian inference. We know that balls can have four sizes: 1, 2, 3 and 4, but ...
- 231
8
votes
0
answers
1k
views
Are Log Predictive Likelihood, Log Predictive Probability, Log Marginal Likelihood and Log Predictive Density same?
I have seen different papers use different terms to express the scoring rules that they used to compare Bayesian models. Some of those terms are,
Log Predictive Density (Bayesian Data Analysis - by ...
- 153
8
votes
1
answer
460
views
Bayesian inference on mean of statistic from population
Suppose that a collection of time intervals $t_i$ have occurred, for $i=1,...,n$. These should be considered as samples from a population governed by some distribution. During these time intervals, ...
- 415
8
votes
0
answers
302
views
Time evolution of a Bayesian posterior
I have a question regarding the time evolution of a quantity related to a Bayesian posterior.
Suppose we have binary parameter space $\{ s_1, s_2 \}$ with prior $(p, 1-p)$,
The data generating ...
- 3,358
8
votes
0
answers
577
views
Dealing with dependent data in a Bayesian model
Background: Consider a series of dependent data points,
$$ y_1,y_2,y_3,\cdots,y_N.
$$
In cases where the dependence is well described by an exponentially decaying
auto-correlation function, it is ...
- 131
8
votes
0
answers
280
views
Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy
I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
- 551
8
votes
0
answers
1k
views
Is my OpenBUGS / WinBUGS model well specified?
I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ...
- 771
8
votes
0
answers
192
views
Question 10.9 from Bayesian Data Analysis, what does accuracy mean here?
I'm doing an independent study in Bayesian Statistics following some chapters from BDA3. When solving the first question from Ch 10 I got stuck. It says:
[If] a scalar variable $\theta$ is ...
8
votes
1
answer
6k
views
Unscented Kalman filter-negative covariance matrix
I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
- 81
8
votes
1
answer
1k
views
Generate Posterior predictive distribution at every step in the MCMC chain for a hierarchical regression model
I'm trying to fit a Bayesian Hierarchical regression model with a random correlated coefficients using R ,I'm using data having 160 groups (schools) to fit a model of math score as a function of one ...
- 1,603
7
votes
1
answer
166
views
Posterior consistency for scale-mixture shrinkage priors in low dimension?
Consider the model [1]
$$y_n=X_n\beta_n+\epsilon_n$$
$$\beta_i|\sigma^2,v_i \sim \mathcal{N}(0,\sigma^2 v_i), i=1,\ldots,p$$
$$v_i \sim \beta^\prime(a,b)$$
$$\sigma^2 \sim \mathcal{IG}(c,d)$$
where $\...
- 129
7
votes
0
answers
144
views
How many sides and what are the probabilities of each side for an unfair die?
Problem
I would like to run an inference that predicts from a series of die rolls, how many sides the die has, and what is the probability of landing on each side.
Example
For example, imagine a die ...
- 71
7
votes
0
answers
703
views
How to explain the difference between confidence and credible interval?
The key difference between Bayesian statistical inference and frequentist statistical methods concerns the nature of the unknown parameters that you are trying to estimate. In the frequentist ...
7
votes
0
answers
426
views
Robust Gamma Regression
I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...
- 223
7
votes
0
answers
193
views
Including feature-dependent priors on output class, in bayesian logistic regression
When doing logistic regression with data $D_N = \{(x_i, y_i)\}_i^N$ with $x_i \in \mathbf{X}^N$ (each data point has N features) and $y_i \in \mathbf{Y}$ being assigned output classes, in a Bayesian ...
- 1,130
7
votes
0
answers
147
views
A puzzling observation by Bradley Efron in his article in Science regarding Bayes’ Theorem in the 21st Century
Mr. Effron has published an interesting article in Science magazine with the enticing title "Bayes'
Theorem in the 21st Century". The article is quite short and can be found here: http://web.ipac....
- 245
7
votes
0
answers
2k
views
How to use LDA to predict topic proportion for new document?
I'm interested to learn how I can use a trained LDA (Latent Dirichlet Allocation) model to make predictions on the topic proportion of a new, unseen document using Naive Bayes.
Let $z \in \{1, 2, ......
- 171
7
votes
1
answer
735
views
Adding a magnitude penalty to a GAM
This is a follow-up to a previous question of mine, explaining the problem in more detail in the hopes of getting more precise advice.
Consider the following structured additive regression model or ...
- 11.1k
7
votes
0
answers
95
views
Bayesian prior over long probability vectors
Suppose you have i.i.d. variables $x_i$ in ${1,\ldots,K}$ modeled as
$$P(x_i = k) = \theta_k$$
and and you want to infer the probability vector $\theta$. A Bayesian approach puts a prior over $\...
7
votes
0
answers
146
views
Bayesian inference via approximate data likelihood
Suppose that we have a very large i.i.d. sample $x_1,...,x_n$ and a data likelihood defined by
$$p(x | \theta,\beta) = \prod_ip(x_i | \theta,\beta)$$. Further suppose that $\theta$ is the parameter ...
- 176
7
votes
0
answers
2k
views
Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization
I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
- 81
7
votes
0
answers
282
views
Cox's Theorem: the necessity of (un)countably additivity
I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
- 551
7
votes
0
answers
3k
views
Help with a proof of Bayes classifier optimality
I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier.
I've never worked through a ...
- 695
7
votes
0
answers
239
views
why use diagonal $\Sigma$ when working with Bayes decision theory?
My prof. said in the class that for Bayes decision rule, the likelihood is Gaussian and in practice, we will almost always work with a diagonal $\Sigma$. Why is that? I know that a diagonal $\Sigma$ ...
- 235
7
votes
1
answer
570
views
Bayesian estimates for Deming regression coinciding with least-squares estimates
Consider the following Deming model with independent replicates :
$$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad
y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, \...
- 19.6k
7
votes
0
answers
170
views
Is this problem Bayesian? And can I use variational approximation?
Suppose there are $N$ samples of observations $\mathbf X(n)$ ($n=1,\cdots,N$), which are given by probability distribution $p(\mathbf X(n)|\mathbf Z(n))$ with their conditions are given by hidden ...
- 71
7
votes
0
answers
2k
views
Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood
I am wondering if I understand those terms correctly. To summarize my thoughts:
In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
user39663
7
votes
3
answers
723
views
Credit Risk and Concentration
I am working with a UK credit-union and we are looking to build a model to assess our credit risk and changes to this over time. We have a number of loans to borrowers who each have a credit rating (...
- 71
7
votes
0
answers
1k
views
Bias Variance tradeoff from a Bayesian perspective
I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
- 131
7
votes
0
answers
2k
views
Combining posterior probabilities from multiple classifiers
I am new to machine learning and can't get my head around this problem. I have two patient datasets, the first ($D_1$) contains $Y,Z,X$ that convey blood-sample information and the second ($D_2$) ...
- 111
7
votes
0
answers
6k
views
Conjugate of Weibull with shape known
This isn't exactly a homework problem but rather a self-selected problem I'm doing to prepare for a midterm.
I can see from Wikipedia that it is an inverse gamma but I am unable to reach the ...
- 1,033
6
votes
1
answer
180
views
No more than $n$ moose, but how many?
Introduction
I am thinking about how to estimate the number of individual moose from wildlife camera photos. I have the latitude and longitude position of each observation, along with a datetime of ...
- 9,522
6
votes
0
answers
248
views
Why does fitting the hyperparameter of Ridge regression at the same time as the model parameters does not lead to a vanishing hyperparameter?
I have been simulating some quadratic data with some noise (constant for all points) into it. I am fitting those data with a polynomial fit with Ridge regression. To find the best hyperparameter, I ...
- 161
6
votes
0
answers
354
views
MCMC - Is Gelman-Rubin diagnostic criteria good for checking convergence of MCMC chains sampling from multimodal distribution?
I am using Bayesian inference to estimate some parameter in a model.
My parameter space is multimodel - most likely bimodal.
Is the Gelman-Rubin diagnostic criteria a good way to check convergence?
...
- 81