Motivation for average log-likelihood In maximum likelihood estimation, I sometimes see the average likelihood function $\frac{1}{N}\mathcal{L}(\theta, x)$ used over the likelihood. Since maximum likelihood estimation optimises over the parameters $\theta$, I dont see the motivation for averaging, since it doesnt change the maximim of $\mathcal{L}(\theta, x)$. Can someone give me some intuition for why we might want to divide by $N$?
 A: One reason to divide by $N$ would be to make numbers comparable over data sets with different sample sizes.  Otherwise it does not seem a very important issue. 
A: TLDR version:
If you are using a first order optimization algorithm, such as gradient ascent, using the average likelihood as your objective function stabilizes the behavior of algorithm as the sample size changes. On the other hand, if you are using a second order optimization algorithm, such as Newton's Method, whether you use likelihood or normalized likelihood is irrelevant. 
More Details
In many problems, the log likelihood can be written as 
$L(\theta|x) = \sum_{i = 1}^n L(\theta|x_i)$
i.e. the log likelihood is the sum of contributions from each observation. In standard gradient ascent, we update $\theta$ with 
$\theta^{(t+1)} = \theta^{(t)} + \alpha \frac{\partial L}{\partial \theta^{(t)}}$
Here, $\alpha$ is often referred to as the "learning rate", and in many problems, is selected without strong justification (i.e. empirically seems to work well, but no proof that is optimal). Choosing a poor learning rate can lead to very poor performance. 
Now, the tricky thing is that we need to realize that 
$\frac{\partial L}{\partial \theta} = \sum_{i = 1}^n \frac{\partial L(\theta | x_i)}{\partial \theta}$
This means that as our sample size gets larger, we take larger steps when using the unnormalized log-likelihood, even when we are the same distance away from the optimal solution. With that in mind, we can see that it should be impossible to pick an $\alpha$ that is optimal for all sample sizes. 
However, if we use the normalized log-likelihood, then our step size should be approximately the same size as the sample size grows. Thus, $\alpha$ is likely to be less sensitive to sample size. 
When using Newton's Method, this is not an issue: the step is
$\theta^{(t+1)} = \theta - H^{-1} \frac{\partial L}{\partial \theta^{(t)}}$
where $H$ is the Hessian. Because the Hessian is affected by the multiplication of normalizing constants in the same way, multiplying the objective function by $1/n$ does not change the step size.
