I am often told that the crucial difficulty in causal inference is that we only observe one value between $Y(1)$ and $Y(0)$ while we want to estimate $E[Y(1) - Y(0)]$. There is always an unobserved value.

Here is my problem: why don't we simply use the samples with treatment $z_i = 1$ to regress $y(1) \sim x$ , and similarly use the samples with treatment $z_i = 0$ to regress $y(0) \sim x$, and combine them to estimate $E[Y(1) - Y(0)]$?

From this perspective, causal inference is just two regression problems and needn't be treated as a special area. I am sure that there must be something wrong, but what is it?