Theoretical grounding for ease of training with a prior If we have a neural network that learns the generative model for P(A, B, C)
And now, we want to learn the generative model for P(A, B, C, D)
Is there any theory that says learning P(A,B,C) and then composing it with P(D | A,B,C) is faster than learning P(A,B,C,D) from scratch?
 A: Probably not. Here is a possible "counterexample".
Let $P(D)$ be some distribution over fixed-length strings which is not too difficult to model. 
Now let $P(A, B, C|D) = I(A,D)I(B,D+1)I(C,D+2)$, where $I(x,y)$ is 1 if $\text{hash}(y) = x$ and otherwise 0. Also fix the output of $\text{hash}$ to be much longer than the input, so the probability of any collision is low. And assume $\text{hash}$ has all the standard cryptographic properties such as pre-image resistance, etc.
Given a decent model of $P(D)$, sampling the joint distribution $P(A,B,C,D)$ is easy. Obtaining an explicit density model is also not hard, since $P(A = x|\text{hash}(D) \neq x) \approx 0$, etc.
On the other hand, modeling $P(A, B, C)$ is much harder, since the output of the hash function will look essentially random. Modeling $P(D|A,B,C)$ is equivalent to  trying to break pre-image resistance, which is also virtually impossible.
A: It's hard to say in general. P(A,B,C,D) has 4 variables so naturally its harder than learning P(A,B,C),  but if you already have P(A,B,C), and need P(D|A,B,C), well running a linear regression to get P(D|A,B,C) is probably gonna be quicker than doing P(A,B,C,D) though a neural network, OTOH, if you did P(D|A,B,C), it might have a lot of singularities and other nasties that make it heard to learn the function. This is why in the EM algorithm they work with P(A,B) rather than P(A|B).  
