1
$\begingroup$

I have implemented a neural network for time series forecasting. The time series consists of count data, so I chose to model it with a negative binomial distribution. My network is an autoregressive model that, given a number of time steps, outputs the mean $\mu$ and dispersion $\theta$ of the negative binomial distribution of the next time step:

$$ \Pr(X = x) = \binom{x+\theta-1}{x} (1-p)^\theta p^x $$ $$ \Pr(X = x) = \binom{x+\theta-1}{x} \left(\frac{\mu}{\theta + \mu}\right)^\theta \left(\frac{\theta}{\theta + \mu}\right)^x $$

To help with training, I want to scale the input data (e.g., divide each element of an input timeseries by the timeseries average value $k$). If I do so, I know I have to multiply the $\mu$ predicted by the network by $k$ to bring the mean back into the original scale. My question is what I have to do with $\theta$ to remove the scaling effect.

$\endgroup$
1
$\begingroup$

After posting my question I realised I know how to recover both the unscaled mean ($\mu$) and variance ($\sigma^2$), which I can then use to compute the unscaled $\theta$. We have $$ \mu = k\mu' \;\;\;\;\;\; \sigma^2 = k^2\sigma'^2 \;\;\;\;\;\; \theta = \frac{\mu^2}{\sigma^2 - \mu} $$ where $\mu'$ and $\sigma'^2$ denote the scaled values.

And doing some substitutions, we obtain $$ \theta = \frac{\mu^2\theta'}{\mu(k -1)\theta' + \mu^2 } $$

$\endgroup$
3
  • $\begingroup$ Had the same problem and was thinking on doing the same calculation, but still don't get where that squared root for the shape parameter in the DeepAR paper comes from. Any luck with that @nicolas? $\endgroup$
    – ERed
    Jul 15 '20 at 21:22
  • 1
    $\begingroup$ @ERed, look at this Github issue. Lorenzo Stella explains the way the shape parameter is upscaled is arbitrary and chosen based on what works best in practice. At the end, what matters is the final network's output (the upscaled one), which is treated as the parameters of a Neg. Binomial. In my understanding, this means the network, until the upscaling, is not really learning a Neg. Binomial., but another "unknown" distribution which after the applied transformation becomes a Neg. Binomial. $\endgroup$
    – nicolas
    Jul 17 '20 at 9:50
  • $\begingroup$ Thank you very much for your answer. I thought that the scaling was happening before and after the training loop, but now I see it is done within the loop to use the proper unscaled negative likelihood. Thank you for the reply. $\endgroup$
    – ERed
    Jul 20 '20 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.