I have a problem in "weather normalization of utility bills" : I would like to find $X$ such as $AX = Y$ where $Y$ is the vector of the monthly Consumption of electricity per day, and $A$ is a matrix of size $(12,3)$. \begin{equation} A = \begin{pmatrix} 1 & CDD_1(T_\alpha)/day & HDD_1(T_\beta)/day \\ \vdots & \vdots & \vdots \\ 1 & CDD_i(T_\alpha)/day & HDD_i(T_\beta)/day \\ \vdots & \vdots & \vdots \\ 1 & CDD_{12}(T_\alpha)/day & HDD_{12}(T_\beta)/day \end{pmatrix} \end{equation} Where
$CDD_i(T_\alpha)/day = \frac{\displaystyle \sum_{j \in days \ in \ month \ i } \left( T_j -T_\alpha \right)^+}{len \ of \ month \ i} $
$HDD_i(T_\beta)/day = \frac{ \displaystyle \sum_{j \in days \ in \ month \ i } \left( T_\beta -T_j \right)^+}{len \ of \ month \ i} $.
Where $T_j = \frac{T_{j , max} + T_{j ,min}}{2}$ is the mean between the maximum and minimum temperature during the day $j$.
When $T_\alpha$ and $T_\beta$ are known for example equal to 65 (Fahrenheit degrees if I consider that people use heating if temperature is less than 65°F and cooling if temperature is more than 65°F), I solve \begin{equation} X= argmin_{z \in \mathbb{R}^3} \| Az - Y \|_2^2 \end{equation} using the normal equation and it's done : I find the $X$ which minimize the sum of the squares of the errors.
For example, I have this bill for the period 2010-12-01 to 2011-11-30. Then $Y$ is the column Consumption/day (see data in csv format at end of question)
I find the following results (module statsmodels for python):
And prediction of Consumption/day is :
[669.49,
675.89,
664.81,
651.66,
624.97,
709.24,
930.76,
1297.32,
1069.53,
857.7,
643.85,
625.63]
Question
If I don't know the values of $T_\alpha$ and $T_\beta$, how can I estimate them ?
It's seems to me that is a kind of problem:
\begin{equation}
X= argmin_{z \in \mathbb{R}^3} \| A_{t_{T_\alpha , T_\beta}}z - Y \|_2^2
\end{equation}
but I have no idea how to solve it.
The only thing I managed to do so far is a double for-loop (on the change point temperatures $T_{\alpha}$ and $T_{\beta}$) and do the regression thereafter to display the adjusted Rsquare in a matrix, but this is not satisfying to me, and this adjusted Rsquare is false due to wrong degrees of freedom.
Data
,Start,End,Comsumption,Nb_days,Consumption/day,CDD65,HDD65,CDD65/day,HDD65/day
1,12/1/2010,12/31/2010,19879,30,662.6,0,921.90,0.00,30.73
2,1/1/2011,1/31/2011,20793,30,693.1,0,999.50,0.00,33.32
3,2/1/2011,2/28/2011,17396,27,644.3,0,778.50,0.00,28.83
4,3/1/2011,3/31/2011,19267,30,642.2,0,705.50,0.00,23.52
5,4/1/2011,4/30/2011,18527,29,638.9,0,368.90,0.00,12.72
6,5/1/2011,5/31/2011,21100,30,703.3,67.4,136.90,2.25,4.56
7,6/1/2011,6/30/2011,26809,29,924.4,210,7.30,7.24,0.25
8,7/1/2011,7/31/2011,39205,30,1306.8,454.2,0.00,15.14,0.00
9,8/1/2011,8/31/2011,32120,30,1070.7,307.2,0.00,10.24,0.00
10,9/1/2011,9/30/2011,24375,29,840.5,163.6,22.70,5.64,0.78
11,10/1/2011,10/31/2011,19325,30,644.2,20.5,225.30,0.68,7.51
12,11/1/2011,11/30/2011,18843,29,649.8,0,376.70,0.00,12.99
