It depends on how your moving average deals with the ends. If there are not $n$ value produced, then if you have a few values from the start of the original series, you ay still be able to do it.
Consider a simple moving average of 5 terms ($s(t) = (y_{t-2}+y_{t-1}+y_{t}+y_{t+1}+y_{t+2})/5$). Imagine we have 10 observations. Then without some rule for $s_1$ and $s_2$ you only have 6 $s$ values ($s_3$ to $s_8$).
You can't get 10 numbers out of 6. If you have a rule to compute $s$ at the ends, or know the original y-values at the ends you may be able to recover the entire series.
Consider it in matrix form: $s = Ay$. Unless $A$ is of full rank, you can't solve for $s$. With the above 5-term moving average, $A$ is of dimension $(n-4)\times n$. You need more information, and knowing the two values at each end (or having some kind of smoothed values for $s$ at the endpoints, as long as the new $A$ has full rank) gives it to you.