How do we understand the relationship between independent probabilities and real-world independence? From what I have come to understand, the events A and B are considered independent for purposes of probability theory when
$$
p(A \cap B) = p(A) \cdot p(B)
$$
Now, supposing I flip two coins. I write down the probabilities for the joint outcomes as $\left(\frac{1}{4}, \frac{1}{4}, \frac{1}{4}, \frac{1}{4}\right)$ and see that the individual coins are independent. But how do we know those probabilities? Well, the first coin could come up either heads or tails with probability $\frac{1}{2}$, then in the case of heads we know that the probabilities for the second coin are still $\frac{1}{2}$… but this seems to be assuming $p(A|B) = p(A)$, which is mathematically just a rearrangement of the above formula, so this feels like circular reasoning.
So, what's the explanation? Why does theoretical independence correspond to practical independence, in a not-apparently-circular way?
 A: I think if you approach it the other way around it is more intuitively understandable. What does it mean that two variables $A$ and $B$ are independent? It means that knowing the probability distribution/function of one tells us absolutely nothing about the other one. The fact that $B$ may or may not have occurred is irrelevant, extraneous, and pretty much distracting when we are thinking of $A$. What does this mean in probabilistic terms? It means that the probability of $A$ occurring, given that $B$ occurred is the same as the probability of $A$ ignoring $B$ completely. We translate this English (or whatever language you wish) concept into symbolic mathematical form as $P(A|B) = P(A)$. This is the root expression, not the product rule, as this is the translation into mathematical symbols of the meaning of the word "independent".
Now that we have defined/translated "independence" to mean $P(A|B) = P(A)$, we can make the following observation:
$$
P(A|B) = P(A)\\
\frac{P(A \cap B)}{P(B)} = P(A)\\
P(A \cap B) = P(A)P(B)
$$
The "product rule" is an outgrowth (via algebraic manipulation) of the definition, not the other way around. 
A: I think there's a problem where you say you "observe the two coins are independent."  You never observe probabilistic independence, per se; it is always a property of the events / random variables under consideration, which are constructed by the modeller to represent some (physical or otherwise) phenomenon.
So, the physical phenomenon here is: there are two fair coins, and tossing either one of them does not affect the outcome of the other.  They're 'independent' in a nontechnical sense, if you want to use that word.
A probabilistic model that we can build to represent this physical phenomenon mathematically is: define two independent, Bernoulli(1/2)-distributed random variables $X_{0}$ and $X_{1}$.  I.e.,
$$
X_{0}, X_{1} \overset{iid}\sim \text{bernoulli}(1/2) \\
$$
Here I mean 'independent' in the technical, probabilistic sense.  That gives us stuff like
$$
P(\{X_{0} = 1\}\cap\{X_{1} = 1\} = P(X_{0} = 1)P(X_{1}=1)
$$
and we can get the joint distribution by enumerating all the combinations of $X_{0}$ and $X_{1}$, as you've done.  Nothing circular to be found.
