Deriving bayes formulas from "Overcoming catastrophic forgetting in neural networks" I am trying to understand the formulas from the paper "Overcoming catastrophic forgetting in neural networks" and am wondering if someone could help explain how they derive these formulas. The paper explains a method for overcoming forgetting of task A when training the same network for task B.
The paper first defines the training of a neural network in a probabilistic way as follows:
$$\log p(\theta \mid D)=\log p(D \mid\theta)+\log p(\theta)-\log p(D)$$
The reasoning they provide for this definition is as follows:
"optimizing the parameters is tantamount to finding their most probable values given some data
$D$. We can compute this conditional probability $p(θ \mid D)$ from the prior probability of the parameters
$p(θ)$ and the probability of the data $p(D \mid θ)$ by using Bayes’ rule"
But since I am not too familiar with Bayes rule this definition is slightly confusing to me how they get this definition. The second formulation they make is to consider $D$ being composed of 2 independent parts, one for task A $(D_A)$ and one for task B $(D_B)$.
$$\log p(\theta \mid D)=\log p(D_B\mid \theta)+\log p(\theta \mid D_A)-\log p(D_B)$$
If anyone could help me fill in the steps / logic behind these formulas that would be very helpful
 A: They got the first equation by taking the log of the normal Bayes rule expression,
$$
p(\theta \mid D) = \frac{p(D \mid \theta) p(\theta)}{p(D)}
\text{.}
$$
To make it clear that this is a general claim (not specific to $D$ and $\theta$), I could rewrite this as
$$
p(B\mid A) = \frac{p(A \mid B) p(B)}{p(A)}
\text{.}
$$
If this is unfamiliar, rearrange these statements about the joint probability drawn from the chain rule $p(A, B) = p(A \mid B) p(B) = p(B\mid A)p(A)$.

Now, we derive the second equation. $D_a$ and $D_b$ are considered independent of each other, so we may write $p(D) = p(D_a)p(D_b)$. They are also conditionally independent given $\theta$, so we may write $p(D \mid \theta) = p(D_a \mid \theta) p(D_b \mid \theta)$.
$$
\begin{align}
\log p(\theta \mid D) &= \log p(D \mid \theta) + \log p(\theta) - \log p(D) & \text{Bayes rule} \\
&= \log p(D_a \mid \theta) + \log p(D_b \mid \theta) + \log p(\theta) - \log p(D_a) - \log p(D_b) &\text{Independence} \\ 
&= \log p(D_b \mid \theta) + \left[\log p(D_a \mid \theta) + \log p(\theta) - \log p(D_a) \right] - \log p(D_b) &\text{Rearrange terms} \\ 
&= \log p(D_b \mid \theta) + \log p(\theta \mid D_a) - \log p(D_b) &\text{Bayes rule again} \\ 
\end{align}
$$
What this tells us is that $D_a$ has updated our prior about $\theta$; we now treat the posterior given $D_a$ as our new prior when we work with $D_b$.
