I agree with Carl that fitting a model to the data would be an ideal solution, if it makes sense in your context. But I also want to suggest that the following way of thinking about your problem might be helpful.
Suppose that I have a time-varying process that looks like a random walk (so that the location is autocorrelated through time). Suppose I measure this position with some measurement error (which is an IID random variable) at equally spaced time points. I understand your question to mean:
How many time points should I average in order to get the best possible information about the process' current location?
If the random walk careens around wildly and the measurement error is small, then the answer might well be use only the most recent point. Using previous points in the average would introduce lots of extra variation due to the random walk, and a single point is already a reasonably good approximation of the process' true location. If the measurement error is large and the random walk moves slowly, then the answer will be use a lot of points. The random walk is relatively stationary, and you need to average over the noise in the measurement. Interestingly, if your measurement noise is extremely fat-tailed (i.e. Cauchy distributed), then the answer will always be use just the most recent point (because the average of multiple points does not provide a better approximation to the central tendency of that distribution than any single point does!).
It should be possible to work out the ideal number of points to use in special cases where the distribution followed by the random walk and the distribution of the measurement error are both known. However, this is precisely the case where a model, as suggested by Carl, would be useful.
Edit Carl's comment also made me realize that it's very likely that a weighted average (that weights more recent points more heavily) could outperform an average that introduces some hard-threshold cutoff for inclusion.