Conditional independence and joint distribution This is something from Chris Bishop's book and I want to confirm that I have understood this correctly.
So, suppose we have a joint distribution between two RVs as:
$$
    P(a, b) = P(a|b)P(b) = P(b|a)P(a)
$$
Now, suppose that $b$ is observed, can I write the joint distribution as:
$$
P(a, b) = P(a|b)P(b) = P(a|b)
$$
Since, $b$ is observed, $P(b)$ is 1.
 A: No, since b is observed doesn't mean that P(b)=1. If you were to repeat the experiment then it wouldn't mean that b will be observed again, because that's what P(b)=1 means. The fact that you observed doesn't make it certain.
A: Let's take $A$ and $B$ to be binary variables to make things simple: $A$ must be equal to either $a_1$ or $a_2$, and $B$ must be equal to either $b_1$ or $b_2$. Suppose that our information state $I$ prior to learning the value of $B$ gives the following joint PDF between $A$ and $B$:
\begin{align*}
P(A=a_1,B=b_1\;|\;I\,) & = p_{11} \\
P(A=a_1,B=b_2\;|\;I\,) & = p_{12} \\
P(A=a_2,B=b_1\;|\;I\,) & = p_{21} \\
P(A=a_2,B=b_2\;|\;I\,) & = p_{22} 
\end{align*}
Now suppose we learn that $B = b_1$. Then we can apply Bayes' Theorem to give the posterior PDF over $A$:
$$
P(A = a_i\;|\;B = b_1,I\,) = \frac{P(A = a_i,B = b_1\;|\;I\,)}{ P(B = b_1\;|\;I\,) }
$$
Note that $P(B = b_1\;|\;I\,)$ is conditioned on our initial information $I$, so it does not necessarily equal $1$. Now we can introduce the probabilities of the joint PDF:
$$
P(A = a_i\;|\;B = b_1,I\,) = \frac{p_{i1}}{p_{11} + p_{21}}
$$
In our posterior information state $\{B = b_1, I\}$, we have $P(B = b_1\;|\;B = b_1, I\,) = 1$, which is quite persuasively evident when one writes it like that! Happily, the posterior probabilities of the states of $A$ sum to $1$:
$$
\sum_{i = 1}^{2} \frac{p_{i1}}{p_{11} + p_{21}} = \frac{p_{11} + p_{21}}{p_{11} + p_{21}} = 1
$$
A: This question is complicated by the choice of notation in the book you are using. With a little more formalism in the notation, the matters that puzzle you are resolved easily.
Consider discrete random variables $X$ and $Y$ with joint distribution
$$
p_{X,Y}(0,0) = P\{X=0, Y=0\} = 0.1, ~~p_{X,Y}(1,0) =P\{X=1, Y=0\} = 0.2,\\
p_{X,Y}(0,1) = P\{X=0, Y=1\} = 0.3, ~~p_{X,Y}(1,1) = P\{X=1, Y=1\} = 0.4.
$$
Then, $X$ is a Bernoulli random variable with parameter $0.6$, that is,
$$p_X(1) = P\{X=1\} = 0.6, ~~p_X(0) = P\{X=0\} = 0.4,$$ while $Y$ is a Bernoulli random variable with parameter $0.7$, that is,
$$p_Y(1) = P\{Y=1\} = 0.7, ~~p_Y(0) = P\{Y=0\} = 0.3.$$  Now, given that $Y=1$, the conditional distribution of $X$ is 
$$p_{X\mid Y=1}(1\mid Y=1) = P\{X = 1\mid Y = 1\} 
= \frac{P\{X=1,Y=1\}}{P\{Y=1\}} = \frac{0.4}{0.7} = \frac 47,\\
p_{X\mid Y=1}(0\mid Y=1) = P\{X = 0\mid Y = 1\} 
= \frac{P\{X=0,Y=1\}}{P\{Y=1\}} = \frac{0.3}{0.7} = \frac 37,$$
that is, conditioned on the event $Y=1$, the conditional distribution of $X$ is Bernoulli with parameter $\frac 47$.  Similarly,  conditioned on the event $Y=0$, the conditional distribution of $X$ is Bernoulli with parameter $\frac 23$.  Work out the details of this last calculation to make sure that you understand.  Then, work out all the details of the calculation that conditioned on $X=1$, the conditional distribution of $Y$ is Bernoulli with parameter $\frac 23$ while conditioned on $X=0$, the conditional distribution of $Y$ is Bernoulli with parameter $\frac 34$.
Next, verify for yourself that for all choices of $i, j \in \{0,1\}$, it is true that
$$p_{X,Y}(i,j) = p_{X\mid Y = j}(i\mid Y=j)p_Y(j) = p_{Y\mid X=i}(j\mid X=i)p_X(i).$$ There is no such thing as $p(a,b) = p(a\mid b)p(b) = p(a\mid b)$ as you think "because $p(b) = 1$ since $b$ has occurred". $p(b)$ does not change its value "when $b$ has occurred" because random variables always occur on each and every trial of the experiment; what changes is the value taken on by the random variable on each trial.
