I am trying to understand the formulas from the paper "Overcoming catastrophic forgetting in neural networks" and am wondering if someone could help explain how they derive these formulas. The paper explains a method for overcoming forgetting of task A when training the same network for task B.
The paper first defines the training of a neural network in a probabilistic way as follows:
$$\log p(\theta \mid D)=\log p(D \mid\theta)+\log p(\theta)-\log p(D)$$
The reasoning they provide for this definition is as follows:
"optimizing the parameters is tantamount to finding their most probable values given some data $D$. We can compute this conditional probability $p(θ \mid D)$ from the prior probability of the parameters $p(θ)$ and the probability of the data $p(D \mid θ)$ by using Bayes’ rule"
But since I am not too familiar with Bayes rule this definition is slightly confusing to me how they get this definition. The second formulation they make is to consider $D$ being composed of 2 independent parts, one for task A $(D_A)$ and one for task B $(D_B)$.
$$\log p(\theta \mid D)=\log p(D_B\mid \theta)+\log p(\theta \mid D_A)-\log p(D_B)$$
If anyone could help me fill in the steps / logic behind these formulas that would be very helpful