# Questions tagged [bayesian]

Bayesian inference is a method of statistical inference that relies on treating the model parameters as random variables and applying Bayes' theorem to deduce subjective probability statements about the parameters or hypotheses, conditional on the observed dataset.

2,381 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
172 views

### What is the "direct likelihood" point of view in statistics?

I am reading a Springer title from 1997 called Applied Generalized Linear Models by James K. Lindsey. In the preface, Lindsey writes For this text, the reader is assumed to have knowledge of basic ...
528 views

### Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2}$$ Here the kernel $K$ is an assigned smooth ...
158 views

### How to construct "reference priors"?

I have been reading about noninformative priors. Two of the most popular priors of this kind seem to be the Jeffreys prior and the reference prior. The Jeffreys prior has a clear construction, being ...
137 views

### Pope effect on pizza - Regression with presence absence and similarity data as dependent variables

I'm trying to figure out the right way to set up a regression when the dependent variables are presence absence data (of pizzas), and the similarity between the present pizzas. Bear with the story: ...
113 views

### Rationale behind Good–Turing frequency estimation?

Good–Turing frequency estimation is a smoothing estimator for estimating a multinomial distribution. It seems very convoluted. From mathematical statistics point of view, what is the rationale ...
1k views

### Fourier transform of a Gaussian process

I would like to discuss and ask a question regarding the Fourier transform of a Gaussian process, if it makes sense. For that purpose, let me describe the following situation. Let $z(s)$ be a ...
643 views

### Density estimation/approximation from MCMC samples

I'm looking to accurately describe the density function of a multivariate posterior probability distribution based on samples from MCMC. As far as I know, in most cases this is done either with a ...
263 views

### Is probability fundamentally about reference classes (real or imagined)?

Question: It seems that frequentism and Bayesianism may not really be different as far as the the ultimate basis for what a probability is (relative frequency within a reference class) - it's just ...
178 views

### Generalization of degrees of freedom for t distribution for coefficients after multiple imputation

Donald Rubin has shown that regression coefficient estimates have fatter tails after multiple imputation and has provided a formula for the degrees of freedom to use as a t-distribution approximation ...
2k views

### Horseshoe priors and random slope/intercept regressions

I'm interested in using the horseshoe prior (or the related hierarchical-shrinkage family of priors) for regression coefficients of a traditional multilevel regression (e.g., random slopes/intercepts)....
290 views

### Model checking in bayesian stats considered "virtually illegal" in the 90's (Andrew Gelman's quote)

In this post, Andrew Gelman says: Bayesian inference can make strong claims, and, without the safety valve of model checking, many of these claims will be ridiculous. To put it another way, ...
596 views

### Hypergeometric: how do I construct a credibility interval around K (population successes) in R?

I have a problem for which I believe I should use the hypergeometric distribution, but I can't figure out how to do it in R. Say I have a bag of marbles with known number ($N$) of marbles, but the ...
10k views

### What does "def" above an equals sign mean?

I am reading this: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf and on equation (17), there is a def on top of the equal sign. What does this mean?
83 views

### Trouble replicating simple example of Bayesian inference

On pages 20-21 of John Kruschke's Doing Bayesian Data Analysis book (2nd ed.), there is an introductory illustration of Bayesian inference. We know that balls can have four sizes: 1, 2, 3 and 4, but ...
2k views

### Implementing Predictive Posterior Distribution Using Stan

Background I had an example that sought to demonstrate the posterior predictive distribution in the context of a normal measurement model. The data that was used is as follows: ...
815 views

### Are Log Predictive Likelihood, Log Predictive Probability, Log Marginal Likelihood and Log Predictive Density same?

I have seen different papers use different terms to express the scoring rules that they used to compare Bayesian models. Some of those terms are, Log Predictive Density (Bayesian Data Analysis - by ...
423 views

### Dealing with dependent data in a Bayesian model

Background: Consider a series of dependent data points, $$y_1,y_2,y_3,\cdots,y_N.$$ In cases where the dependence is well described by an exponentially decaying auto-correlation function, it is ...
149 views

### Question 10.9 from Bayesian Data Analysis, what does accuracy mean here?

I'm doing an independent study in Bayesian Statistics following some chapters from BDA3. When solving the first question from Ch 10 I got stuck. It says: [If] a scalar variable $\theta$ is ...
5k views

### Unscented Kalman filter-negative covariance matrix

I have recently started working on the unscented Kalman filter. I coded the numerically stable version (i.e., square root Kalman filter) and used MATLAB for implementing. In the final update step, ...
93 views

### Why does Quadratic (Normal/Laplace) Approximation fail on multilevel models?

In Statistical Rethinking, 2nd Edition, section 13.1, Richard McElreath says: Why doesn’t simple quadratic approximation, using for example quap, work with multilevel models? When a prior is itself a ...
144 views

### Why is BART so accurate in causal inference?

The famous paper Dorie,2017 shows that BART performs dramatically well in causal inference. In my replication, MSE in BART can be 40% lower than MSE in other machine learning methods. But all machine ...
271 views

### Robust Gamma Regression

I am modeling some spectroscopic data where the response of the instrument to the size of the input is strictly positive and non-linear. Gamma regression seems like a good choice to explain the data, ...
7k views

### Singular fit with simplest random structure in lmer (lme4), is a Bayesian approach the only option?

I'm running a mixed model with the lmer function from the lme4 package in R and ran into some issues with singular fits. I get the warning message 'singular fit', ...
176 views

### Including feature-dependent priors on output class, in bayesian logistic regression

When doing logistic regression with data $D_N = \{(x_i, y_i)\}_i^N$ with $x_i \in \mathbf{X}^N$ (each data point has N features) and $y_i \in \mathbf{Y}$ being assigned output classes, in a Bayesian ...
113 views

### A puzzling observation by Bradley Efron in his article in Science regarding Bayes’ Theorem in the 21st Century

Mr. Effron has published an interesting article in Science magazine with the enticing title "Bayes' Theorem in the 21st Century". The article is quite short and can be found here: http://web.ipac....
2k views

### Mean, median, or mode of skewed posterior?

I'm estimating an ICC from 2 and 3-level hierarchical models using rstanarm. The simplest models are: y ~ (1|group) or ...
360 views

### Bayesian inference on mean of statistic from population

Suppose that a collection of time intervals $t_i$ have occurred, for $i=1,...,n$. These should be considered as samples from a population governed by some distribution. During these time intervals, ...
1k views

139 views

### Bayesian inference via approximate data likelihood

Suppose that we have a very large i.i.d. sample $x_1,...,x_n$ and a data likelihood defined by $$p(x | \theta,\beta) = \prod_ip(x_i | \theta,\beta)$$. Further suppose that $\theta$ is the parameter ...
1k views

### Dealing with auxiliary random variables for Mean-Field Variational Inference in Bayesian Poisson factorization

I am studying as a part of a class assignment a recent paper on Poisson factorization. Some points of the paper regarding the usage of some auxiliary variables are not clear to me. I would like to ...
221 views

### Cox's Theorem: ignorance, objective priors, and the Mind Projection Fallacy

I've been trying to understand Cox's Theorem and the problems surrounding it. There's so much information on this topic that I've become confused as to the exact state of the theorem. I've gathered ...
152 views

### why use diagonal $\Sigma$ when working with Bayes decision theory?

My prof. said in the class that for Bayes decision rule, the likelihood is Gaussian and in practice, we will almost always work with a diagonal $\Sigma$. Why is that? I know that a diagonal $\Sigma$ ...
1k views

### Is my OpenBUGS / WinBUGS model well specified?

I've just started trying to use OpenBUGS for Bayesian analysis of stochastic volatility models. In particular, I'm trying to calculate stochastic covariance, similar to the DC-MSV model specified by ...
158 views

### Is this problem Bayesian? And can I use variational approximation?

Suppose there are $N$ samples of observations $\mathbf X(n)$ ($n=1,\cdots,N$), which are given by probability distribution $p(\mathbf X(n)|\mathbf Z(n))$ with their conditions are given by hidden ...
2k views

### Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
573 views

### Credit Risk and Concentration

I am working with a UK credit-union and we are looking to build a model to assess our credit risk and changes to this over time. We have a number of loans to borrowers who each have a credit rating (...
1k views

### Predict random effects in a multilevel model with Empirical Bayes

In multilevel models, it is possible to predict (not estimate) the random effects by Empirical Bayes after the model parameters have been estimated. I know how to use the ...
1k views

### Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
1k views

### Combining posterior probabilities from multiple classifiers

I am new to machine learning and can't get my head around this problem. I have two patient datasets, the first ($D_1$) contains $Y,Z,X$ that convey blood-sample information and the second ($D_2$) ...
5k views

### Conjugate of Weibull with shape known

This isn't exactly a homework problem but rather a self-selected problem I'm doing to prepare for a midterm. I can see from Wikipedia that it is an inverse gamma but I am unable to reach the ...
829 views

### Generate Posterior predictive distribution at every step in the MCMC chain for a hierarchical regression model

I'm trying to fit a Bayesian Hierarchical regression model with a random correlated coefficients using R ,I'm using data having 160 groups (schools) to fit a model of math score as a function of one ...
52 views

### How do we derive the conditional mode as the solution to linear regression, for uniform cost function?

I know that if the cost functions are respectively the least squares ($L^2$) and the absolute deviation ($L^1$), the solution to linear regression is the conditional mean and the conditional median ...
101 views

### Official name of a common type of Bayesian simulation study

There is a kind of simulation study that is commonly used to validate an implementation of a Bayesian model: For independent replication $i = 1, ..., n$: Draw a set of "true" parameters ...
158 views

### Why does fitting the hyperparameter of Ridge regression at the same time as the model parameters does not lead to a vanishing hyperparameter?

I have been simulating some quadratic data with some noise (constant for all points) into it. I am fitting those data with a polynomial fit with Ridge regression. To find the best hyperparameter, I ...
The $\ell_1$ norm of model parameters is often added to loss functions because it induces sparsity in the solution of the overall cost function: $$c(\theta) = \log L(x|\theta) + \lambda ||\theta||_1$$...