Calculating Probability based on Linear Regression in R or Excel Suppose we have a monthly budget of 1,000. For every day of the month we keep track of the total (cumulative) expenses. For example, let's say we are on 15th November and we have the following vector for these 15 days.
cumulative_expenses <- c(100,120,140,170,230,240,260,260,260,300,350,430,440,460,470).
What I want is to calculate the probability of going over the budget of 1,000 in the end of the month.
I can predict the total expenses for the rest of the month using a linear regression model. But how could I calculate the probability of going over this budget, either in R or Excel?
In the end, I would like to have this probability for every day of the month, so that I know on each day in the range of the month (let's say from the 2nd day until the 29th) what is the probability of going over the budget.
 A: Implicitly, your OLS linear regression will be assuming that the true values of the target will be normally distributed around the predictions the regressor makes, with a variance of $\sigma ^{2}$. It just so happens, that you do not need to know the value of $\sigma ^{2}$ in order to find the optimal regression parameters.
People tend to say that the linear regression cost function is
$C = \sum_{i=1}^{N}(y_{i} - f(x_{i}; \theta))^{2}$
where $f(x; \theta)$ is your regressor function and $\theta$ are the parameters which you optimise your loss wrt.
The justification for this usually is that you are actually optimising the following cost function (known as the likelihood)
$C = \prod _{i=1}^{N} \left(\frac{1}{\sqrt{2\pi \sigma^{2}}}e^{-\frac{1}{2\sigma ^{2}}\left(y_{i} - f(x_{i};\theta)^{2}\right)}\right)$
It just so happens that if you find the $\theta$ which minimise the former, you will also maximise the latter.
To see this, take the logarithm of the latter and throw away constant terms to obtain:
$-\frac{N}{2}\ln \sigma^{2} - \frac{1}{2\sigma ^{2}}\sum_{i=1}^{N}(y_{i} - f(x_{i};\theta))^{2}$
so minimising the sum will maximise the logarithm of the original cost function and thus maximise the original cost function. Differentiating this and setting to zero, we find that
$\sigma^2 = \frac{1}{N}\sum_{i=1}^{N}(y_{i} - f(x_{i};\theta))$
So to tie this all together:
1: Perform your linear regression as you would normally, to find your optimal $\theta$
2: Plug this optimal $\theta$ into $\sigma^2 = \frac{1}{N}\sum_{i=1}^{N}(y_{i} - f(x_{i};\theta))$ to calculate your optimal $\sigma ^{2}$
3: You wanted to know your probability of going over budget at the end of the month. Firstly make your prediction for the end of the month, using the regressor. Let's say it says "the single most likely value is that you'll be at 750". Now let's say that at stage 2 you calculated to $\sigma^{2}=10,000$ and thus $\sigma=100$. Well going over 1000 would correspond to a Z-score of (250)/100=2.5, so you can now consult your z-table and you'd fine a 0.0062% chance.
I'd just add that this is a so-called maximum likelihood approach. There are more principled/complex approaches for approaching problems like this called Bayesian approaches, but this is probably a good place to start.
