I have the below equation:

Y' = Y + β Y1 + β^2 Y2 + β^3 Y3

This is time-series data where

Y' = prediction year
Y = latest year (most current data available)
Y1 = Y - 1
Y2 = Y - 2
Y3 = Y - 3

Formulation of problem statement: We don't have data for Y'. To train β, we have taken Y1, Y2, Y3 as training vector. Y as result for training vector.

I am trying to learn β which optimizes MSE(Mean squared error) using Stochastic Gradient Descent (both equations below):


$$ \frac{1}{2n} \sum_{n}\left( \beta \cdot Y_1 + \beta^2\cdot Y_2 + \beta^3\cdot Y_3 - Y \right)^2 $$

Stochastic Gradient Descent Beta update equation:

$$ \beta^{new} = \beta^{old} - \eta \frac{\partial \epsilon}{\partial \beta} \\ where\ \frac{\partial \epsilon}{\partial \beta} = \frac{1}{n} \sum \left( \beta \cdot Y_1 + \beta^2\cdot Y_2 + \beta^3\cdot Y_3 - Y \right) \left( Y_1 + 2\beta\cdot Y_2 + 3\beta^2\cdot Y_3 \right) $$

Using this method, I am getting stuck at local optima. ( and sometimes I get math error due to nan and inf, -inf).

How do I learn β using this optimization problem ? Or is there a better method of learning β for this problem statement ?

  • $\begingroup$ Isn't that a least-squares problem you are solving? It looks like an autoregressive AR(4) model which can be estimated by ordinary least squares (OLS). At the same time, it does not look like exponential smoothing, so I wonder why the tag? (Or does this AR(4) model have an equivalent exponential smoothing representation?) $\endgroup$ – Richard Hardy Nov 20 '16 at 9:43
  • $\begingroup$ @RichardHardy I wanted to just optimize a Beta for the first equation that I have written. That equation is a variance of Exponential Smoothing. I have the same Alpha for all our data, so I replaced 1-Alpha by Beta and I am trying to optimize that. The way I am doing it is by minimizing the MSE. I wanted to know if that is the best way to optimize Beta? Also I am getting stuck at local maximas of Beta. I are unable to reach the global maxima. Every time I run this, I get a different value of Beta. $\endgroup$ – thenakulchawla Nov 20 '16 at 22:59
  • $\begingroup$ I see. I don't have any good ideas beyond what I have written above, i.e. that it is an OLS problem and as such is very well known and deeply studied, so there already are solutions to it. $\endgroup$ – Richard Hardy Nov 21 '16 at 6:19
  • $\begingroup$ Thanks a lot. I wasn't normalizing my beta values, therefore it wasn't getting optimized. $\endgroup$ – thenakulchawla Nov 24 '16 at 21:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.