It depends on how your moving average deals with the ends. If there are not $n$ values produced, then if you have a few values from the start of the original series, you may still be able to do it.
Consider a simple moving average of 5 terms ($s(t) = (y_{t-2}+y_{t-1}+y_{t}+y_{t+1}+y_{t+2})/5$). Imagine we have 10 observations. Then without some rule for $s_1$ and $s_2$ (and $s_9$ and $s_{10}$, you only have 6 $s$ values ($s_3$ to $s_8$).
You can't get 10 numbers out of 6. If you had a rule to compute $s$ at the ends that gave $s_1,...,s_n$ but relied only on $y_1,...,y_n$, or know the original y-values at the ends you may be able to recover the entire series.
Consider it in matrix form: $s = Ay$. Unless $A$ is of full rank, you can't solve for $s$. With the above 5-term moving average, $A$ is of dimension $(n-4)\times n$. You need more information, and knowing the two values at each end (or having some kind of smoothed values for $s$ at the endpoints, as long as the new $A$ has full rank) gives it to you.
Edit:
Note that in our example of the 5-term simple moving average that
$5[s(t-2)-s(t-3)] = y(t)-y(t-5)$
so
$y(t) = 5[s(t-2)-s(t-3)] +y(t-5)$
Which means that you don't need values at both ends -- if you have the first handful of values of $y$ you can reconstruct the rest (indeed, you could do it from the final values as well, in similar fashion)
Going backwards tends to be much less stable. If you just "guess" some y-values, some experimentation suggests that (under the sorts of series I was generating, which were from local level models, the kind of thing where it might make sense to use a moving average) you seem to be better off representing the original series by the smoothed value.
I have had some interesting results from a combination of shifting the recovered $y$ and averaging with the smooth, which seems to be able to recover a little better than the smooth, but I assume that much better can be done
Response to Q in comments:
And if instead of fully reconstructing it, which is impossible to me given I don't have enough info, how about to estimate it?
If we attempt to estimate the whole series, I think that estimation would depend on some assumptions about the form of model for $E(y_t)$ and the distribution of error about it. I'd be first inclined to consider trying to use the E-M algorithm on such a problem, but there are other approaches as well.
I don't know how successful these would be, though, but a suitable choice of model may do quite well.
It occurs to me that deconvolution methods are probably the way to approach this, but I haven't really done anything with those.