20
votes
Accepted
Probabilistic programming vs "traditional" ML
It's generally true in my personal experience as a professional data scientist.
It's true in my personal experience because it's what I observe most of the time. If you're asking why it happens this ...
- 12.1k
17
votes
Accepted
COVID in Germany, LOO-CV for time series
Overview quick remarks
The model with three points does make a better fit.
The fit with three points is only slightly better.
The model with only one point is not very bad. The difference in loocv ...
- 71.9k
14
votes
Why are there recommendations against using Jeffreys or entropy based priors for MCMC samplers?
This is of course a diverse set of people with a range of opinions getting together and writing a wiki. I summarize I know/understand with some commentary:
Choosing your prior based on computational ...
- 30.5k
10
votes
Accepted
What is pm.Potential in PyMC3?
We use pm.Potential here primarily to get around the definition of a likelihood. We ordinarily use it to constrain our likelihood in the manner described in the ...
- 116
9
votes
Accepted
What would be the reason that the posterior distribution looks like the prior using MCMC
Assuming your likelihood and MCMC work correctly, which we can't know of course, and assuming your new variables are not ineffective:
If you increase the number of parameters under calibration with ...
- 7,739
9
votes
Accepted
Why are there recommendations against using Jeffreys or entropy based priors for MCMC samplers?
They do not provide any scientific/mathematical justification for doing so. Most of the developers do not work on this kind of priors, and they prefer to use more pragmatic/heuristic priors, such as ...
- 106
8
votes
Accepted
What to take in consideration when we use Bayesian Methods on Big Data problems?
Author here. There are a few points I can elaborate on:
Your data is going to be distributed across many computers in a real cluster, so each computer has a fraction of the data. Locally, then, the ...
- 12.1k
7
votes
Accepted
Bayesian recurrent neural network with keras and pymc3/edward
From a pure implementation perspective, it should be straightforward: take your model code, replace every trainable Variable creation with ed.Normal(...) or sth ...
- 13.7k
7
votes
Accepted
pymc3: acceptance probabilities and divergencies after tuning
You might have better luck on our discourse: https://discourse.pymc.io/
A couple of notes: You need to use: pm.sample(..., nuts_kwargs=dict(target_accept=0.95)) ...
- 1,076
7
votes
Accepted
Why use MCMC sampling when using conjugate priors?
You are correct that if you have a conjugate prior, there's no need to use MCMC as the posterior has a closed form solution. MCMC tutorials that present a problem where we know the posterior already ...
- 20.2k
5
votes
Accepted
Optimize starting parameters for Bayesian Linear Regression?
I'll illustrate my answer with a simple example. Imagine that your data $X_1,\dots,X_n$ are counts that follow a Poisson distribution. Poisson distributtion is described using a single parameter $\...
Tim♦
- 136k
5
votes
Accepted
Why can't PyMC3 fit a uniform distribution with a Normal prior?
I think the program has done exactly what you asked it to and has done so pretty well (from the limited information you show). Given the information that you are 100% certain that this is data from a ...
- 30.5k
5
votes
PyMC beginner: how to actually sample from the fitted model
Landed here several years later when looking for the same thing using PyMC3, so I am going to leave an answer relevant to the new version: (from Posterior Predictive Checks).
...
- 11.2k
5
votes
What is pm.Potential in PyMC3?
There is a description of potentials in the old version of PyMC documentation:
http://pymc-devs.github.io/pymc/modelbuilding.html#the-potential-class
From what I understand, probabilistic ...
- 826
5
votes
Probabilistic programming vs "traditional" ML
To combat ShadowTalker above's point about probabilistic ML being not quite up to snuff yet, is definitely true as-is, but there have been some really exciting advances in scalability and complexity ...
- 405
5
votes
Accepted
Relationship between laplace and l1 regularization
There are four points of improvement to make the relationship between the l1 regularization and the Bayesian MAP estimate equivalent.
1. Slightly different definitions of $\lambda$
The optimization ...
- 71.9k
4
votes
Using empirical priors in PyMC
If you already have a prior $p(\theta)$ and a likelihood $p(x|\theta)$, then you can easily find the posterior $p(\theta|x)$ by multiplying these and normalizing:
$$p(\theta|x)=\frac{p(\theta)p(x|\...
- 496
4
votes
Accepted
Bayesian modeling of train wait times: The model definition
I will tell you first what I would do and then I'll answer the specific questions you had.
What I would do (at least initially)
Here is what I gather from your post, you have training waiting times ...
- 4,631
4
votes
Why can't PyMC3 fit a uniform distribution with a Normal prior?
I can't really read the code, but using a normal model to approximate a uniform should always result in exactly this result: a normal centered at the uniform mean with variance to match. If you want a ...
- 89.2k
4
votes
How to build a PyMC model to detect multiple 'switch points'?
It's been three years, but I believe this might be the approach mentioned in the comments by @twiecki. It uses a truncated Dirichlet Process Mixture Model to detect multiple change points without any ...
- 41
4
votes
Fitting simple (binomial) model in PyMC - slow convergence
Here is my shot at the problem in PyMC3. I can be wrong how the model is built, so please correct me where I am wrong.
The data are 50 observations (50 binomial draws) that are i.i.d. This ...
- 2,505
4
votes
Accepted
sampling behind bayesian hierarchical models
Here is the basic structure of a hierarchical model. In order to simplify the exposition, I'm going to modify the notation a bit.
Let there be $n$ groups (or units), $Y = (Y_1, \ldots, Y_n)$, where $...
- 3,226
4
votes
Accepted
How to interpret posterior distribution plots for multiple priors?
I would say the model you present has one prior for multiple parameters (in this case two: $\mu$ and $\theta$). Because there are two parameters, the prior is a joint prior. Similarly, there is a ...
- 3,226
4
votes
Accepted
Distorted hyperpriors when sampling from the prior only
I assume the problem is that the MCMC sampler finds it difficult to sample from the joint posterior distribution of $v$, $\mu$ and $\sigma$. This kind of problem has been described before for the kind ...
- 30.5k
4
votes
Accepted
Spline regression via PyMC3
Here is a minimal example, which works for a DataFrame df with columns X and Y. It uses <...
- 1,197
4
votes
Reason behind only using internal knots when defining basis splines
Presumably because the upper and lower boundaries of the data are easily identified from the data themselves, whereas, in those two examples, the authors are wanting to specify the (internal) knot ...
- 45.8k
4
votes
Accepted
How to interpret rank bar plot of a MCMC trace?
From the dox:
From the paper: Rank plots are histograms of the ranked posterior
draws (ranked over all chains) plotted separately for each chain. If
all of the chains are targeting the same posterior,...
- 20.2k
3
votes
Accepted
Outlier detection in beta distributions
A more systematic way to deal with this problem would be to use an explicit mixture model, with a specification of the distribution of the 'outliers'. A simple form would be to use a mixture of a ...
- 118k
3
votes
Accepted
Multilevel model with partially pooled variance
The general idea of your model can be handled in Stan (and possibly in PyMC and other statistical inference packages). Here are a couple of suggestions:
Hyperparameters
We (i.e. Andrew) have been ...
- 326
3
votes
Regression Mixture in PYMC3
An alternative is to use the marginalized mixture model (see also this SO answer). This utilizes the NUTS using ADVI and converges within 6000 samples.
...
- 233
Only top scored, non community-wiki answers of a minimum length are eligible