This is a great question (+1, where does it come from?), and your final result $0$ is correct. However, your notation and reasoning are hard to follow/comment because they are largely non-standard. So I would like to post a new answer that follows the conventional measure-theoretic conditional expectation framework.
Intuition
To begin with, it is helpful to study patterns of the function $f(x) = \frac{|x^2 - 1|}{4}$ and the function $g(x) = x^3 - 3x$ on the domain $x \in [-2, 2]$. The plots of $f$ and $g$ are shown as follows:
From the chart above, it is clear that
- $\max\limits_{x \in [-2, 2]} |g(x)| = 2$;
- For any $y \in (-2, 2)$, the equation $g(x) = y$ always has three real roots in $[-2, 2]$. In addition, if denote the three roots from left to right by $x_1, x_2, x_3$ respectively, then $x_1 \in [-2, -1], x_2 \in [-1, 1], x_3 \in [1, 2]$.
In addition, since the quadratic, linear and constant coefficients of the cubic polynomial $x^3 - 3x - y$ are $0, -3$ and $-y$ respectively, it follows by Vieta's formula that
\begin{align*}
& x_1 + x_2 + x_3 = 0, \tag{1}\label{1} \\
& x_1x_2 + x_1x_3 + x_2x_3 = -3, \tag{2}\label{2} \\
& x_1x_2x_3 = y. \tag{3}\label{3}
\end{align*}
Now for a given $y \in (-2, 2)$, suppose that we have observed $X^3 - 3X = y$, then $X$ has to be one of the three roots $x_1, x_2, x_3$. In view of the distribution pattern of $x_i$'s (i.e., the second bullet point above) and $P(-2 \leq X \leq -1) = P(-1 \leq X \leq 1) = P(1 \leq X \leq 2) = \frac{1}{3}$, it follows by $\eqref{1}$ that
\begin{align*}
E[X\mid X^3 - 3X = y] = \frac{1}{3}x_1 + \frac{1}{3}x_2 + \frac{1}{3}x_3 = 0. \tag{4}\label{4}
\end{align*}
Since $\eqref{4}$ holds for all $y$, we can conclude that
\begin{align*}
E[X\mid X^3 - 3X] = 0. \tag{5}\label{5}
\end{align*}
Rigorous Proof
To validate the intuitive answer $\eqref{5}$, we need to examine that $0$ satisfies the two defining properties of the conditional expectation:
- $0$ is $\sigma(X^3 - 3X)$-measurable;
- For any $G$ in $\sigma(X^3 - 3X)$, the functional equation holds:
\begin{align*}
\int_G X ~\mathrm dP = \int_G 0 ~\mathrm dP = 0.
\end{align*}
Bullet point 1 holds trivially. To verify bullet point 2, note that since the sub-$\sigma$-field $\sigma(X^3 - 3X)$ is generated by sets $\{\{X^3 - 3X \leq y\}: y \in \mathbb{R}\}$, it suffices to show that for any $y \in \mathbb{R}$,
\begin{align*}
\int_{X^3 - 3X ~\leq~ y} X ~\mathrm dP = \int_{\{x \in [-2, 2]:\; x^3 - 3x~ \leq ~y\}}xf(x)~\mathrm dx = 0, \tag{6}\label{6}
\end{align*}
where the first equality in $\eqref{6}$ follows from the change-of-variable formula.
When $y \geq 2$,
\begin{align*}
\int_{\{x \in [-2, 2]:\; x^3 - 3x ~\leq~ y\}}xf(x)~\mathrm dx = \int_{-2}^2 xf(x)~\mathrm dx = 0,
\end{align*}
hence $\eqref{6}$ holds.
When $y \leq -2$,
\begin{align*}
\int_{\{x \in [-2, 2]:\; x^3 - 3x ~\leq~ y\}}xf(x)~\mathrm dx = \int_\varnothing xf(x)~\mathrm dx = 0,
\end{align*}
hence $\eqref{6}$ holds.
When $y \in (-2, 2)$, as in the Intuition block, let $x_1 < x_2 < x_3$ be three roots of the equation $x^3 - 3x = y$. It then follows by $x_1 \in [-2, -1], x_2 \in [-1, 1]$ and $x_3 \in [1, 2]$ that
\begin{align*}
& \int_{\{x \in [-2, 2]:\; x^3 - 3x~ \leq~ y\}}xf(x)~\mathrm dx \\
=& \int_{-2}^{x_1}xf(x)~\mathrm dx + \int_{x_2}^{x_3}xf(x)~\mathrm dx \\
=& \frac{1}{4}\int_{-2}^{x_1}x(x^2 - 1)~\mathrm dx + \frac{1}{4}\int_{x_2}^1x(1 - x^2)~\mathrm dx +
\frac{1}{4}\int_{1}^{x_3}x(x^2 - 1)~\mathrm dx \\
=& \frac{1}{16}\left[(x_1^4 + x_2^4 + x_3^4) - 2(x_1^2 + x_2^2 + x_3^2)\right] - \frac{3}{8} \\
=& \frac{1}{16}\left[(x_1^2 + x_2^2 + x_3^2) + y(x_1 + x_2 + x_3)\right] - \frac{3}{8} \tag{7}\label{7} \\
=& \frac{1}{16}(x_1^2 + x_2^2 + x_3^2) - \frac{3}{8} \tag{8}\label{8} \\
=& \frac{1}{16}[(x_1 + x_2 + x_3)^2 - 2(x_1x_2 + x_1x_3 + x_2x_3)] - \frac{3}{8} \\
=& \frac{1}{16}[0 - 2 \times (-3)] - \frac{3}{8} \tag{9}\label{9} \\
=&~ 0.
\end{align*}
This shows $\eqref{6}$ holds for $y \in (-2, 2)$. In the calculation above, $\eqref{8}$ used identity $\eqref{1}$, $\eqref{9}$ used identities $\eqref{1}$ and $\eqref{2}$. $\eqref{7}$ is a consequence of
\begin{align*}
& x_1^4 + x_2^4 + x_3^4 - 3(x_1^2 + x_2^2 + x_3^2) \\
=& x_1(x_1^3 - 3x_1) + x_2(x_2^3 - 3x_2) + x_3(x_3^3 - 3x_3) \\
=& x_1y + x_2y + x_3y = y(x_1 + x_2 + x_3).
\end{align*}
This completes the proof.
