# Parameter estimation under Gamma noise distribution

I have a model as follows:

$$y_i = \theta x_i + \eta_i, i=1,2,...,N$$

where $$y_i$$ and $$x_i$$ are known observations greater than $$0$$, the $$\eta_i\sim Gamma(a,b), a>1$$. Now, I want to obtain the model parameter $$\theta$$ using the maximum likelihood estimation. First, the likelihood function is given as:

$$L(\theta ) = {\left( {\frac{{{b^a}}}{{\Gamma (a)}}} \right)^n}.\prod\limits_{i = 1}^N {({y_i} - \theta {x_i})^{\alpha-1}} .\exp [ - b\sum\limits_{i = 1}^N {({y_i} - \theta {x_i}} )]$$

then logarithmic

$$\log L(\theta ) \propto (\alpha-1)\log \left( {\prod\limits_{i = 1}^N {({y_i} - \theta {x_i})} } \right) - b\sum\limits_{i = 1}^N {({y_i} - \theta {x_i}} )]$$

the first derivative of $$\theta$$ is

$$\frac{{\partial \log(L)}}{{\partial \theta }} = (\alpha-1) \sum\limits_{i = 1}^N {(\frac{{ - {x_i}}}{{{y_i} - \theta {x_i}}}} ) + b\sum\limits_{i = 1}^N {{x_i}}$$

Obviously, estimation of $$\theta$$ cannot be directly obtained from the equation $$\frac{{\partial L}}{{\partial \theta }}=0$$ due to the complex expression.

To solve this problem, the folloing methods are considered:

(1) Can $$\sum\limits_{i = 1}^N {(\frac{{ - {x_i}}}{{{y_i} - \theta {x_i}}}} )$$ be accurately approximated as a simple expression.

(2) Can translate the problem of $$\max(\log L(\theta))$$ into a simple optimization problem without using differential method.

How to get the estimation of $$\theta$$ ? Hope to give some suggestions, thank you very much

The likelihood is missing the term $$\alpha-1$$, because the gamma PDF contains $$x^{\alpha-1}$$: $$-\log L(\theta ) \propto -(\alpha-1)\sum\limits_{i = 1}^N {\log {({y_i} - \theta {x_i})} } + b\sum\limits_{i = 1}^N {({y_i} - \theta {x_i}} )$$
Minimizing this expression is a convex optimization problem with the constraints $$y_i-\theta x_i>0$$. The second part is affine, and the first part is affine composition of a convex function ($$-\log x$$). Sum of two convex functions is also convex. So, this problem can be solved easily via a CVX solver. Or you can even apply gradient descent yourself.