I think the problem here is that different people mean different things when they talk about probabilistic programming. However, it seems to me that there are basically two different views.
On the one hand, the term is used for modelling and inference with bayesian networks. And I have the feeling, that this is a view adopted by many data scientists. This could be called probabilisitic programming in a narrow sense, since it's focused more or less on a specific class of models.
On the other hand there is a community that is actively involved in the development of probabilistic programming languages, or better, probabilistic programming systems. Their goal is to develop formal languages to describe models in probabilistic terms, and these models no longer have to be bayesian networks or conform to other statistical models. Instead, they are totally general, so everything that you can write down in a "normal" programming language would also work in one of those languages. The probabilistic programming system also has an inference engine, that is capable of doing inference on such models. This could be called probabilistic programming in a wide sense.
A recent publication that gives a thorough introduction and also makes the distinction between graphical models and general probabilistic programming is this:
Meent, Jan-Willem van de, Brooks Paige, Hongseok Yang, and Frank Wood. An Introduction to Probabilistic Programming. ArXiv:1809.10756 [Cs, Stat], September 27, 2018. http://arxiv.org/abs/1809.10756.