0
$\begingroup$

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$

What priors should I use for updating these parameters ($\theta_{1},\theta_{2},\theta_{3}$) in a Gibbs sampler to accommodate the conditions on Taylor expansion? Any help would be appreciated! Thank you so much in advance

$\endgroup$
1
$\begingroup$

Note that your problem is a linear model if you assume the noise is Gaussian. To specify the model you should specify this noise. I assume it is Gaussian. Then it sounds natural to use a Normal prior.

I would most simply use a Normal distribution with mean 0 and covariance matrix $I_3$ as a prior. It works like a basic $L_2$ regularization with coefficient 1 (the non-Bayesian equivalent of a prior). It says "all coefficients are around 0 with variance 1".

But before doing so, you must normalize your input ($y-\alpha$) and output (whatever you call it) to be between say [-1;1] (or have mean 0 variance 1) to avoid "scale issues". You may also normalise $(y-\alpha)$ and $(y-\alpha)^2$ separately but it's unnecessary imo.

"Scale issues" happen when for example $y$ has order of magnitude $10^{-6}$ because some choice of units (measuring a microscopic distance in meters for example). Then a natural order of magnitude for $\theta_2$ would be $10^6$ times $\theta_1$ for the same effect on the output. For $\theta_3$ it would be $10^{12}$ times... The covariance matrix of the prior would need to take this into account which I find cumbersome. Normalization hopefully bypass all these problems.

$\endgroup$
  • $\begingroup$ Thank you for your helpful answer dear @Benoit, how can i normalize $(y-\alpha)$ in a Gibbs sampler? $\endgroup$ – Afshin Oct 27 '17 at 19:46
  • $\begingroup$ I would normalize the dataset before running the sampler. Write $y'=a(y-\alpha)+b$ with the appropriate constants Same for the output. Then run the sampler on $y'$, get samples $(\theta_1',\theta_2',\theta_3')$ and convert to them back to $(\theta_1,\theta_2,\theta_3)$. $\endgroup$ – Benoit Sanchez Oct 28 '17 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.