1
$\begingroup$

I have a textfile with the two columns

$$\mathbf{x}=(x_1,...,x_i)$$

$$\mathbf{y}=(y_1,...,y_i)$$

I want to use the following model for the data

$$y_i=A\sin\left(\frac{x_i}{B}\right)+C\epsilon_i,$$

where $\epsilon_i\sim N(0,1)$ and independent.

By guessing I found that $A=5.2, \ B=5.3$ and $C=1.0$ gives me a pretty good fit. Now I want to write a function in R-code that computes the likelihood function (the probability of observed data $y_1,\ldots,y_{i}$ given the observed values $x_1,...,x_i$ and the observed values for the parameters). But before I do that, I need to understand what's going on mathematically.

The posterior is given by

$$\pi(x_1,\ldots,x_i\mid y_1,\ldots,y_i)=\frac{\color{red}{\pi(y_1,\ldots ,y_i\mid x_1,\ldots,x_i)} \cdot \pi(x_1,\ldots,x_i)}{\pi(y_1,\ldots,y_i)},$$

where $\pi(y_1,\ldots,y_i\mid x_1,\ldots,x_i)$ is the likelihood. But how do I calculate the, posterior, evidence and the prior here? Any help is appreciated.

$\endgroup$
6
  • $\begingroup$ The prior needs to be chosen by you, and you should choose it wisely (non-informative is the term). The posterior is then obtained by looking at a table of Conjugate-priors (if you chose poorly you are screwed as far as doing this by hand goes). $\endgroup$ Dec 5, 2018 at 13:46
  • 1
    $\begingroup$ If you assume the $\epsilon_i$ are independent then you can immediately write down the likelihood, purely mechanically, by applying the definitions of "$N(0,1)$" and independence. This has nothing to do with prior or posterior distributions. $\endgroup$
    – whuber
    Dec 5, 2018 at 14:25
  • $\begingroup$ @user2974951 - So If I want to use a flat prior, can I just let it be proportional to a constant? However, I still don't have my posterior and evidence, so I cant solve for the red colored part yet. $\endgroup$
    – Parseval
    Dec 5, 2018 at 14:50
  • $\begingroup$ @whuber - I forgot to add in the question that $\epsilon_i$ are independent, will edit. I'd be very glad if you could show me how to do this, I have not been able to find a similar example anywhere. Maybe you can link me somewhere where they show some example? $\endgroup$
    – Parseval
    Dec 5, 2018 at 14:52
  • 1
    $\begingroup$ This is a regression problem, so this search will turn up comparable examples: stats.stackexchange.com/…. The first hit arguably is a duplicate: stats.stackexchange.com/questions/47040/…. The answer at stats.stackexchange.com/questions/305908/… also works, even though it's an overly contorted account of something that is basically simple (IMHO). $\endgroup$
    – whuber
    Dec 5, 2018 at 15:18

1 Answer 1

2
$\begingroup$

You've written down the wrong expression for the posterior to do Bayesian regression. Consider the $\vec{x}$ to be fixed by design. Thus, you don't need a prior for the $x$. Rather, you need to set a prior for A, B, and C. Define $r = y - \hat{y}$ in the usual way. The likelihood, based on the probability model for $\epsilon$ and given A, B, and C would be:

$$L(y_i | A, B, C)= \prod_{i=1}^n \phi\left(\left(y_i - A \sin(x_i/B)\right)/C\right)$$

where $\phi$ is the standard normal density.

$\endgroup$
6
  • $\begingroup$ Thanks for this answer, I still don't understand a few things here: (1) In terms of what variable is the normal density expressed? (2) Why did you divide $y-\hat{y}$ by $C$ in the expression of the likelihood? (3) Why does one need to define an $r$? I can't find anyhting about that in this wikipedia article: en.wikipedia.org/wiki/Bayesian_linear_regression $\endgroup$
    – Parseval
    Dec 5, 2018 at 15:48
  • $\begingroup$ @Parseval 1) The normal probability model is for the residuals. 2) because C is a dispersion parameter and doing this gives standard normal RVs. 3) so one can calculate the likelihood. The term $r$ is a convenient shorthand for $Y-\mathbf{X}\beta$ in the linear regression case. you're not doing linear regression. Rather you're doing spectral regression, since the sinusoidal trend in X predicts the Y. But no matter, subtract observed and expected and get a residual. $\endgroup$
    – AdamO
    Dec 5, 2018 at 16:44
  • $\begingroup$ @AdanO - Sorry for wasting your time with the stupid questions. Everything makes sense now. Thank you sir. $\endgroup$
    – Parseval
    Dec 5, 2018 at 17:17
  • 1
    $\begingroup$ @Parseval don't be rough on yourself. When sorting things out in the comments, I tend toward terseness so as to be clearer and more direct. $\endgroup$
    – AdamO
    Dec 5, 2018 at 17:28
  • 2
    $\begingroup$ The likelihood is missing a $C^{-n}$ factor. $\endgroup$
    – Xi'an
    Dec 6, 2018 at 16:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.