I often deal with distributions of outcome variables that are highly skewed (power-law-like), and I typically want to predict these outcomes.
I am wondering when it is appropriate to take a log of the outcome variable and apply OLS, and when it is appropriate to use negative binomial regression?
Assuming you've already identified the predictor(s) of interest/importance, the considerations for a model (in approximate order of importance) would be:
a. do you want to model the conditional mean or something else (e.g. a quantile? some more robust location-estimate than the mean? ...)?
b. What is the anticipated form of the relationship between response and predictors (linear? exponential? power? unknown-but-smooth? unknown but smooth and monotonic? etc etc) ...
should the variability be expected to be fairly constant? changing with mean? changing but unrelated to mean? ...
do you anticipate any substantial dependence between observations?
Now comes distributional considerations and whether you want to worry about bounding impact of influential points (note we haven't yet looked at our data). But the distribution you need to worry about is the conditional distribution (distribution at given value(s) of predictors) not the marginal distribution.
Note that taking logs and fitting OLS would be ideal (close to optimal) if the relationship is exponential and your conditional response is very close to lognormal - which is continuous while the negative binomial is discrete. Further note that negative binomial models have a nonzero probability of a 0 but you can't take log of 0.
In practice, aside from discreteness / issues with small counts there may sometimes be little else to distinguish the two. Here's a plot of (conditionally) negative binomial (left) and lognormal (right) response, both with log "link". As you see they look pretty similar - they have similar movement of the mean with $x$, similar spread at each $x$ and so on.
On the other hand if you consider a lognormal model, why not a gamma or Weibull, for example? It's fairly easy to fit a model with the same kind of linear-in-the-logs relationship with any of them.
One thing that is different is the way the observations enter the model -- if at the optimum we approximate both models with transformed weighted least squares linear model, the points will get somewhat different relative weight.
Something to keep in mind is if you're transforming, the predicted value on the log scale (which is an estimate of the mean of the logs) will not represent a mean when you transform back. If the conditional variability is very small that may not bother you, but more typically it can make a big difference -- in particular $\exp(E(\log(Y|x)))<E(Y|x)$ when the variance is non-zero.
