For the past year, I've been hearing a lot about Probabilistic Programming (PP) frameworks like PyMC3 and Stan, and how great PP is. And today, someone shared this link with me: Pyro: a Deep Probabilistic Programming Language
However, I don't really follow what is special about it since it feels like whatever you can do in PP you can do in any other general purpose language. I'm sure there are technical aspects about PP that make it attractive (e.g. parallel computing), but this aside, is PP really any different from any other language?
Question: I was wondering if there was a consensus of what PP was and how it differs from other statistical-focused software like R, Matlab, Mathematica. It should be noted that
Stan are focused on more Bayesian analysis.
Doing a little research on Google, I came across the following two definitions. The first more abstract, and the second more about PP's technical characteristics.
1.2. Probabilistic Programming Is
Instead, probabilistic programming is a tool for statistical modeling. The idea is to borrow lessons from the world of programming languages and apply them to the problems of designing and using statistical models. Experts construct statistical models already—by hand, in mathematical notation on paper—but it's an expert-only process that's hard to support with mechanical reasoning. The key insight in PP is that statistical modeling can, when you do it enough, start to feel a lot like programming. If we make the leap and actually use a real language for our modeling, many new tools become feasible. We can start to automate the tasks that used to justify writing a paper for each instance.
Here's a second definition: a probabilistic programming language is an ordinary programming language with
randand a great big pile of related tools that help you understand the program's statistical behavior.
Both of these definitions are accurate. They just emphasize different angles on the same core idea. Which one makes sense to you will depend on what you want to use PP for. But don't get distracted by the fact that PPL programs look a lot like ordinary software implementations, where the goal is to run the program and get some kind of output. The goal in PP is analysis, not execution (added emphasis).
I'd like to know if the general statistical community agrees with these two definitions of PP, and if there are any other characteristics this definition may be missing.