Bayesian & Power-Law Distributions

Question

I'm looking to conduct a Bayesian regression analysis with three independent variables (all roughly normally distributed) and a single dependent variable (job performance) that follows a power-law distribution. Instead of assuming that job performance follows a normal distribution, how can I specify that it follows a power-law distribution in R?

Answer 1

There is an important implication if your independent variable follows a power-law distribution -- namely the size of your sample data and whether it is large enough to be reliable for model estimation. The usual "rules of thumb" about sample size don't apply to many heavy tailed distributions, esp. power law distribution with low value of the exponent parameter $\alpha$.

Here is a blog post that gives a tutorial on this topic: http://exploringpossibilityspace.blogspot.com/2013/08/tutorial-how-fat-tailed-probability.html

Therefore, it would be prudent for you to try to fit a distribution to the data so you can evaluate the adequacy of your sample size. The R package poweRlaw is very good for this purpose.

Answer 2

As far as I can see, the distribution of the IVs isn't directly relevant.

Assuming you're talking about a continuous power-law distribution ($f(y)\propto y^{-k}$ for all $y>1$), this would be a Pareto distribution.

One approach would be to model $y^*=\ln(y)$ as exponential using a GLM, where the power in the Pareto becomes the rate parameter, as long as the available link functions for it allow you to match the model you want. Failing that, you will likely need to look at writing and dealing with the likelihood more directly for whatever model you have.

In packages that don't offer the exponential directly, this can be done with a gamma GLM, setting the dispersion parameter to 1.

--

In R you can fit an exponential GLM as follows:

fit <- glm(formula =...,  family = Gamma)
summary(fit,dispersion=1)   

If you specifically want a Bayesian GLM, see the arm package. You may also find the following document helpful - Bayesian Generalized Linear Models in R, Starkweather, 2011 (pdf).