Skip to main content
20 votes
Accepted

Probabilistic programming vs "traditional" ML

It's generally true in my personal experience as a professional data scientist. It's true in my personal experience because it's what I observe most of the time. If you're asking why it happens this ...
shadowtalker's user avatar
  • 12.8k
17 votes
Accepted

COVID in Germany, LOO-CV for time series

Overview quick remarks The model with three points does make a better fit. The fit with three points is only slightly better. The model with only one point is not very bad. The difference in loocv ...
Sextus Empiricus's user avatar
14 votes

Why are there recommendations against using Jeffreys or entropy based priors for MCMC samplers?

This is of course a diverse set of people with a range of opinions getting together and writing a wiki. I summarize I know/understand with some commentary: Choosing your prior based on computational ...
Björn's user avatar
  • 33.5k
10 votes
Accepted

What is pm.Potential in PyMC3?

We use pm.Potential here primarily to get around the definition of a likelihood. We ordinarily use it to constrain our likelihood in the manner described in the ...
Max Margenot's user avatar
9 votes
Accepted

Why are there recommendations against using Jeffreys or entropy based priors for MCMC samplers?

They do not provide any scientific/mathematical justification for doing so. Most of the developers do not work on this kind of priors, and they prefer to use more pragmatic/heuristic priors, such as ...
Prior's user avatar
  • 106
8 votes
Accepted

What to take in consideration when we use Bayesian Methods on Big Data problems?

Author here. There are a few points I can elaborate on: Your data is going to be distributed across many computers in a real cluster, so each computer has a fraction of the data. Locally, then, the ...
Cam.Davidson.Pilon's user avatar
7 votes
Accepted

Bayesian recurrent neural network with keras and pymc3/edward

From a pure implementation perspective, it should be straightforward: take your model code, replace every trainable Variable creation with ed.Normal(...) or sth ...
bayerj's user avatar
  • 13.8k
7 votes
Accepted

pymc3: acceptance probabilities and divergencies after tuning

You might have better luck on our discourse: https://discourse.pymc.io/ A couple of notes: You need to use: pm.sample(..., nuts_kwargs=dict(target_accept=0.95)) ...
twiecki's user avatar
  • 1,076
7 votes
Accepted

Why use MCMC sampling when using conjugate priors?

You are correct that if you have a conjugate prior, there's no need to use MCMC as the posterior has a closed form solution. MCMC tutorials that present a problem where we know the posterior already ...
Cliff AB's user avatar
  • 21.4k
5 votes

What is pm.Potential in PyMC3?

There is a description of potentials in the old version of PyMC documentation: http://pymc-devs.github.io/pymc/modelbuilding.html#the-potential-class From what I understand, probabilistic ...
JPN's user avatar
  • 826
5 votes

PyMC beginner: how to actually sample from the fitted model

Landed here several years later when looking for the same thing using PyMC3, so I am going to leave an answer relevant to the new version: (from Posterior Predictive Checks). ...
Jan Kukacka's user avatar
  • 11.5k
5 votes
Accepted

Optimize starting parameters for Bayesian Linear Regression?

I'll illustrate my answer with a simple example. Imagine that your data $X_1,\dots,X_n$ are counts that follow a Poisson distribution. Poisson distributtion is described using a single parameter $\...
Tim's user avatar
  • 140k
5 votes

Probabilistic programming vs "traditional" ML

To combat ShadowTalker above's point about probabilistic ML being not quite up to snuff yet, is definitely true as-is, but there have been some really exciting advances in scalability and complexity ...
JoeTheShmoe's user avatar
5 votes
Accepted

Relationship between laplace and l1 regularization

There are four points of improvement to make the relationship between the l1 regularization and the Bayesian MAP estimate equivalent. 1. Slightly different definitions of $\lambda$ The optimization ...
Sextus Empiricus's user avatar
4 votes

Using empirical priors in PyMC

If you already have a prior $p(\theta)$ and a likelihood $p(x|\theta)$, then you can easily find the posterior $p(\theta|x)$ by multiplying these and normalizing: $$p(\theta|x)=\frac{p(\theta)p(x|\...
Andris Birkmanis's user avatar
4 votes

How to build a PyMC model to detect multiple 'switch points'?

It's been three years, but I believe this might be the approach mentioned in the comments by @twiecki. It uses a truncated Dirichlet Process Mixture Model to detect multiple change points without any ...
smba's user avatar
  • 41
4 votes

Fitting simple (binomial) model in PyMC - slow convergence

Here is my shot at the problem in PyMC3. I can be wrong how the model is built, so please correct me where I am wrong. The data are 50 observations (50 binomial draws) that are i.i.d. This ...
Vladislavs Dovgalecs's user avatar
4 votes
Accepted

Notation in mixed effects models

This looks like very similar notation to that used in other mixed effects packages. There are 2 random intercepts, for stimulus and ...
Robert Long's user avatar
  • 63.9k
4 votes
Accepted

sampling behind bayesian hierarchical models

Here is the basic structure of a hierarchical model. In order to simplify the exposition, I'm going to modify the notation a bit. Let there be $n$ groups (or units), $Y = (Y_1, \ldots, Y_n)$, where $...
mef's user avatar
  • 3,236
4 votes
Accepted

How to interpret posterior distribution plots for multiple priors?

I would say the model you present has one prior for multiple parameters (in this case two: $\mu$ and $\theta$). Because there are two parameters, the prior is a joint prior. Similarly, there is a ...
mef's user avatar
  • 3,236
4 votes
Accepted

Distorted hyperpriors when sampling from the prior only

I assume the problem is that the MCMC sampler finds it difficult to sample from the joint posterior distribution of $v$, $\mu$ and $\sigma$. This kind of problem has been described before for the kind ...
Björn's user avatar
  • 33.5k
4 votes

Is it always a requirement to declare a distribution model first before applying MCMC models/bayesian analysis?

Yes, you need to assume some priors, but you can build up hierarchical models in pymc. Pairwise or more combinations of these very well known distributions might lead to highly sophisticated and ...
gunes's user avatar
  • 57.8k
4 votes
Accepted

Finding the Poisson rate parameter with PyMC3

First of all, you have far too many chains for a problem like this. Second of all, your tuning parameter is far too high. Something along the lines of ...
Demetri Pananos's user avatar
4 votes
Accepted

Bayesian liability threshold model

If the threshold is zero and you integrate $\psi$ out of the posterior, the second model becomes a probit model. Gibbs samplers actually estimate the $\psi$ parameters, but that is not a good idea (if ...
Ben Goodrich's user avatar
  • 2,008
4 votes
Accepted

Structural equation models without circular definition of latent variables

Assuming $(\mathbf{I} - \mathbf{B})$ is invertible, we can rewrite the $\boldsymbol{\eta}$ equation as follows: $$ \begin{align} \boldsymbol{\eta} &= \boldsymbol{\alpha}+\mathbf{B}\boldsymbol{\...
Jake Westfall's user avatar
4 votes
Accepted

Spline regression via PyMC3

Here is a minimal example, which works for a DataFrame df with columns X and Y. It uses <...
Abraham D Flaxman's user avatar
4 votes

Reason behind only using internal knots when defining basis splines

Presumably because the upper and lower boundaries of the data are easily identified from the data themselves, whereas, in those two examples, the authors are wanting to specify the (internal) knot ...
Gavin Simpson's user avatar
4 votes
Accepted

How to interpret rank bar plot of a MCMC trace?

From the dox: From the paper: Rank plots are histograms of the ranked posterior draws (ranked over all chains) plotted separately for each chain. If all of the chains are targeting the same posterior,...
Taylor's user avatar
  • 21.2k
3 votes

How can I include sample size information in my Bayesian inference model?

I think I should add some explanation. This example was to show estimates with small samples sizes can have extreme values. I think if I was to "model" this example, I would model it as follows: For ...
Cam.Davidson.Pilon's user avatar
3 votes

Regression Mixture in PYMC3

An alternative is to use the marginalized mixture model (see also this SO answer). This utilizes the NUTS using ADVI and converges within 6000 samples. ...
LmW.'s user avatar
  • 233

Only top scored, non community-wiki answers of a minimum length are eligible