Figuring out which delta to use when calculating standard deviation So I'm working on something for work where I'm trying to refine how we do our financial projections and one of the key components of that is determining monthly unit sales, specifically, what we're projecting at for the end of the month.
So the most standard, back-of-the-envelope way to do this would simply be taking the total number of sales you have on day X, and then multiplying that by the remaining percentage (ex: day 15 of a 30 day month would be 30-15/15) and then adding those two numbers together (so if you're at 100 sales on the 15th you would be on pace to finish with 200 sales).
As I said, that's a pretty simplistic way of doing it. So I'm trying to refine that methodology and one of the first things I'm doing is seeing how well pacing performs historically. Say we use the example above where on the 15th we were pacing towards 200, but we actually finished with 225 sales in the month, which means the delta would be -25. But lets say instead we were pacing towards 250 sales on the 15th, meaning that the delta would be +25.
My question is, when I try to build a standard deviation off this as a first step, taking everyday of the month and seeing how the pace factored against the final result, which delta should I be looking at that will give me better accuracy in forecast my end of the month projections. Should I only be looking at how far off the pace was from the final result as a positive integer (so my 200 and 250 examples would both be evaluated as 25) or should I also be factoring in the over/under (so my 200 and 250 example would be evaluated as -25 and 25).
 A: Short answer is to compare methods, I think I would look at something like mean absolute deviation? or mean root squared deviation?
Let $y_t$ be your actual sales in month $t$. Let $\hat{y}_t$ be your forecast sales. The difference between an actual value and a forecast value in the context of regression analysis is called a residual. In the most general sense, it is called a deviation.
$$\epsilon_t = y_t - \hat{y}_t$$
You could also take the logarithm of $y_t$ and $\hat{y}_t$ and then  difference would loosely be the percent difference:
$$ \text{alternative: } {\epsilon}_t = \log{y_t} - \log{\hat{y}_t} \approx \frac{y_t - \hat{y}_t}{\hat{y}_t}$$
Typically what one does is apply some function to the residual $\epsilon_t$ and then sum up over across all observations to come to some notion of the overall loss.
Examples of different ways to aggregate:
Sum of absolute deviations (where $|x|$ denotes the absolute value of $x$):
$$ \sum_{t=1}^T \left| \epsilon_t \right| $$
Could normalize by $T$ to get Mean absolute deviation:
$$ \frac{1}{T} \sum_{t=1}^T \left| \epsilon_t \right| $$
Sum of squared residuals:
$$ \sum_{t=1}^T \epsilon_t^2$$
Could use the root mean squared deviation:
$$ \sqrt{ \frac{\sum_{t=1}^T\epsilon_t^2}{T} }$$
In the context sales, there may be asymmetry in that a too low forecast may be worse than a too high forecast. If that's true, you could create an asymmetric loss function:
$$L(x) = \left\{ \begin{array}{rl} x: & \text{ if $x \geq 0$}\\-10x: & \text{ if $x < 0$} \end{array}  \right\} $$
Then sum up that:
$$\sum_{t=1}^T L(\epsilon_t) $$
Going further: find parameter values that minimize loss
Warning: this is getting a bit more stream of consciousness...
Your forecast will be a function of some parameters $\theta$ and information $x_t$ available at time $t$:
$$\hat{y_t} = f(x_t, \theta)$$
Hence you could write the total loss as a function of $\theta$. Eg. with the sum of squared residuals, the overall loss might be:
$$ \mathcal{L}(\theta) = \sum_t \left( y_t - f(x_t, \theta) \right) ^2$$
One could choose parameter values $\theta$ to minimize this:
$$ \text{minimize (over $\theta$)} \sum_t \left( y_t - f(x_t, \theta) \right) ^2$$
If your forecast function is linear and $x_t$ and $\theta$ are vectors:
$$f(x_t, \theta) = x_t \cdot \theta $$
then the whole problem is ordinary least squares regression.
\sum
