6
$\begingroup$

According to this wikipedia article, score is the derivative of the log-likelihood function. However, I don't understand what if we have two parameters? For example, the logarithm of pdf has the following form: $$ln f(x|a,b) = a\cdot g(x) + b \cdot h(x) + \phi(a,b)$$ I think that this is the sum of partial derivatives i.e.

$$s = g(x) + h(x) + \phi(a,b)_{a}' + \phi(a,b)_{b}'$$

but I'm not sure.

$\endgroup$
9
$\begingroup$

The score for a multiple parameter problem (a vector parameter) is itself a vector. We need to take partial derivatives of the log likelihood with respect to each model parameter.

Let's consider an example. Find the score vector for $X_1, X_2, \ldots, X_n \sim N(\mu, \sigma^2)$ where the $X_i$ are iid $N(\mu, \sigma^2)$ samples. Let $\mathbf{x} = (x_1, x_2, \ldots, x_n)$ be a random sample of $(X_1, X_2, \ldots, X_n)$.

In this case, it can be shown that the log likelihood is

$$\ell(\mu, \sigma | \mathbf{x}) = -\frac{n}{2}\log (2 \pi) - n\log\sigma - \frac{1}{2\sigma^2} \sum_{i=1}^n (x_i - \mu)^2$$

So differentiation w.r.t. $\mu$ gives

$$ \frac{\partial \ell}{\partial \mu} = \frac{1}{\sigma^2} \sum_{i=1}^n (x_i - \mu)$$ differentiation w.r.t. $\sigma$ gives $$ \frac{\partial \ell}{\partial \sigma} = -\frac{n}{\sigma} + \frac{1}{\sigma^3}\sum_{i=1}^n (x_i - \mu)^2.$$

The score is then the vector $S(\mu, \sigma) = \left( \frac{\partial \ell}{\partial \mu}, \frac{\partial \ell}{\partial \sigma} \right)^T$.

Concisely, we can write $S(\mathbf{\theta}) = \nabla_{\mathbf{\theta}} \ell(\mathbf{\theta}|\mathbf{x})$ for any vector parameter $\mathbf{\theta}$ where $\ell(\theta |\mathbf{x})$ is the log likelihood function.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.