1
$\begingroup$

It was suggested I ask this here instead of Stack Overflow.

I am trying to plot the negative log likelihood of an exponential distribution. I am not getting how I am supposed to think of it. The equation for negative log likelihood is provided...

$$l(\lambda; x_1, \dots, x_n) = n \ln(\lambda) - \lambda \sum_{j=1}^n x_j$$

I am not getting how I am supposed to plot it when there is only one output. How is this used in relation to gradient descent? Is this function performed multiple times and the rate parameter changes? I have looked online and cannot find anything I can really understand.

$\endgroup$
1
  • 2
    $\begingroup$ The data $x_1, \cdots, x_n$ are treated as known constants. These values never change for the given model. The parameter $\lambda$ is the primary argument of the function. For this model $\lambda>0$, hence you just need to code a function that takes two arguments $\lambda$, which changes, and $x_1, \cdots, x_n$, which are held fixed. $\endgroup$
    – user277126
    Feb 27, 2022 at 1:48

1 Answer 1

1
$\begingroup$

You have some numpy vector, x. Take the sum.

S = np.sum(x)

You have the sample size, n, by taking the length of x.

n = len(x) # maybe something related to x.shape

Now evelyate the function over a spectrum of lambda values, such as every $0.1$ from $-5$ to $+5$. I’ll call the variable “lamb” since “lambda” has a meaning in Python.

lamb = np.arange(-5, 5.01, 0.1)
L = n * np.log(lamb) - lamb * S

Plot it!

plt.scatter(x, L)

The likelihood function is just a function of your lambda values.

To find the maximum likelihood estimator of $\lambda$, determine the $\lambda$ that maximizes this function.

SPOILERS BELOW

(Don’t read ahead until you’ve figured out the bug in the above code (or at least have given it an attempt and think you’ve figured it out).)

You can’t take a logarithm of a number unless they it positive, so you would have to define your lamb vector as:

lamb = np.arange(0.1, 5.01, 0.1)

(Yes, complex logarithms exist.)

$\endgroup$
7
  • $\begingroup$ There’s a subtle bug in the code that will keep it from compiling. This was $100\%$ accidental, but im $\endgroup$
    – Dave
    Feb 27, 2022 at 2:30
  • $\begingroup$ Thank your for the help! This helped, but when it comes to the gradient descent part, please tell me if my understanding is correct: I want to take the derivative of the function. Then, either increase/decrease the lambda (from the func) to decrease the slope of the tangent line? When it's 0 I've arrived at my MLE? $\endgroup$
    – Vinny
    Feb 27, 2022 at 17:35
  • $\begingroup$ Yes, the slope tells you the direction to move to get higher. If the slope is positive, move to the right. If the slope is negative, move to the left. $\endgroup$
    – Dave
    Feb 27, 2022 at 17:39
  • $\begingroup$ Thank you for your help! Mind helping with the derivative? When I look it up it seems that the derivative is (1/lambda) + (1/(lamdba^2))*(Sum of rand var X's). Is this incorrect? Should I just use a specific function? $\endgroup$
    – Vinny
    Feb 27, 2022 at 19:41
  • $\begingroup$ If $f(x)=a\log(x)-bx$, what is $\frac{df(x)}{dx}?$ $\endgroup$
    – Dave
    Feb 27, 2022 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.