I want to construct my likelihood.
General case:
If my data do come from a line of the form $y = mx + b$ and the uncertainties are normally distributed with mean zero and known variance $\sigma_y^2$, then the likelihood would be: $$p(y|x, \sigma_y, m, b) = \frac{1}{\sqrt{2\pi \sigma_y^2}} \exp(-\frac{(y - mx - b)^2}{2\sigma^2}) $$

However, in my case, the errors follow a gamma distribution. How can I construct my likelihood?


To make things clearer, I have a set of data points represented by $x_i$ and $y_i$ that have measurement errors denoted by $\sigma_{x,i}$ and $\sigma_{y,i}$.
Example, my data points look the following:
$x$ $\sigma_x$ $y$ $\sigma_y$
-0.5 $\pm$ 0.02 0.14 $\pm$ 0.004
0.2 $\pm$ 0.03 0.5 $\pm$ 0.002
I want to calculate the likelihood $p(y|x, \sigma_y, m, b)$. To check if I can use the formula written above I had to test whether the measurement uncertainties ($\sigma_y$) are Gaussian. I did a QQ plot and the result was that they don't.
The measurement uncertainties seem to come from a Gamma distribution.
My question is how can I model the likelihood if the measurement errors come from a Gamma distribution.

  • $\begingroup$ How did you arrive at this equation to begin with? What is the definition of likelihood? $\endgroup$ – jlimahaverford Mar 18 '15 at 12:57
  • $\begingroup$ @jlimahaverford I explained in more details the equation of the likelihood in this question $\endgroup$ – aloha Mar 18 '15 at 13:05
  • $\begingroup$ This question needs clarification. Normally, we would think of something as an "error" only if its conditional mean is zero. Thus it appears you are assuming that the distribution of $y$ conditional on $x$ has some kind of Gamma distribution whose mean is $mx+b$. Is that your intention? And how many parameters do you include in your Gammas: one (for the shape), two (for shape and scale), or even three (for shape, scale, and an additive locational offset)? For the three-parameter Gamma an alternative interpretation is that $mx+b$ is the third parameter. $\endgroup$ – whuber Mar 18 '15 at 21:39
  • $\begingroup$ Rather than trying to do it with errors, define the conditional distribution of your response given your predictors. $\endgroup$ – Glen_b -Reinstate Monica Mar 19 '15 at 5:52
  • $\begingroup$ Do you have a compelling rationale for using a Gamma distribution of your errors? I would agree with @Glen_b since what you're really interested in is the conditional model of your outcome variable given explanatory variables. In Econometrics you should be able to give some sort of interpretation of why exactly gamma distributed errors and not say normal, which is more conventional. Also you can always use the model equation to solve for your errors as outlined in the answer below. Cool question though! $\endgroup$ – Hirek Mar 20 '15 at 12:32

If we use the probability density function (pdf) of the Gamma distribution given in here then the pdf of uncertainties or ($b$) is $$f(b;k,\theta)=\dfrac{b^{k-1}e^{-\frac{b}{\theta}}}{\theta^k\Gamma(k)} \quad (1)$$ To write the loglikelihood of $y$ you neeed to find the pdf of $y$. The easiest way is to use the distribution function as follows: $$P(y\leq z)=F_{y}(z)=P(mx+b\leq z)=P(b\leq z-mx)=F_{b}(z-mx).$$ Now take the derivative w.r.t $z$ from both side to find the pdf of $y$ evaluated at $z$ i.e. $$f_{y}(z)=\dfrac{dF_{y}(z)}{dz}=\dfrac{dF_{b}(z-mx)}{dz}=f_{b}(z-mx). \dfrac{d (z-mx) }{dz}=f_{b}(z-mx) \quad (2).$$ Next in (1), replace $b$ with $z-mx$ to have: $$f_{y}(z;x,m,k,\theta)=\dfrac{(z-mx)^{k-1}e^{-\frac{(z-mx)}{\theta}}}{\theta^k\Gamma(k)}; z>0, k>0, \theta >0 \quad (3).$$ Eq. (3) is the loglikelihood of $y$ when $b$ has a Gamma distribution with parameters $k$ and $\theta$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.