The Bayesian Information Criterion (BIC) is proportional to the log of the maximised likelihood. The likelihood is a density with units given by the inverse units of the parameters.

We are free to set the units of the parameters. For example in a linear regression using millilitres (ml) of rainfall to predict height of plants in meters (m), the parameter associated with rainfall has units m/ml.

Therefore, we can also change the units of the likelihood. In this example we could measure rain in litres (l) instead. The units of the likelihood function would change from ml/m to l/m.

The value of the maximum likelihood will depend on the units in which it is specified. Hence, by changing the units of the parameters, we can change the value of the maximum likelihood to be anything we like.

As a result, using the BIC to compare models with different parameters is useless, as all results depend on the units used in each model. Where have I gone wrong? Thank you.

  • 5
    $\begingroup$ One of the prerequisites for direct use of information criteria for model comparison is that the dependent variable is exactly the same between the different models. Different units make it different. $\endgroup$ – Richard Hardy Feb 20 '18 at 14:08
  • $\begingroup$ A slight aside: I thought a probability density was a unitless quantity as it is the same as a "density" in physics or calculus, an infinitesimal quantity. I do not think this fact relates to whether common units implies BIC is appropriate for comparing models with different parameters; it is probably neither a sufficient or necessary condition. $\endgroup$ – AdamO Feb 20 '18 at 15:03
  • $\begingroup$ What do you mean by "changing the units of parameters"? Do you mean regressors in a multivariate model? Most inference, including the likelihood ratio, is invariant to changes of scale in these parameters. $\endgroup$ – AdamO Feb 20 '18 at 15:05
  • $\begingroup$ Thanks for your comments. @RichardHardy even if we keep the dependent variable in the same units, wont the units of the likelihood function still depend on the units of the independent variables? The independent variables (and their units) will vary across models. $\endgroup$ – Josh12 Feb 20 '18 at 17:06
  • $\begingroup$ @AdamO isn't it the case that if we integrate the likelihood function across all parameter space we will end up with a probability which has no units? Therefore the original density that was integrated over must have units given by the inverse of the units of the parameters. When I say change units of the parameters I mean changing the units of the predictors, yep. In a regression this would change the units of the associated parameters. Why would the likelihood ratio be invariant to this? We can construct exactly the same argument used against the BIC for the likelihood ratio. Thank you! $\endgroup$ – Josh12 Feb 20 '18 at 17:09

A year on and I have realised my confusion was due to a basic mathematics misunderstanding. I was worried that you could change the height of the likelihood function by changing the units of predictors in your model.

Consider the likelihood for a parameter with units of millilitres per litre (ml/m). Imagine this likelihood was Gaussian. I thought that if you changed the units of the parameter to say, litres per meter (l/m), then the height of the likelihood would change. However, the height does not change. To see this, imagine changing the variable used for our Gaussian likelihood. Once the variable has been changed the height of the Gaussian will remain the same.

  • 1
    $\begingroup$ This is incorrect, because the likelihood does depend on the units. However, the differences that one employs in comparing BIC values do not depend on the units, because a change in units merely adds a constant to the log likelihood. $\endgroup$ – whuber Jul 29 '19 at 13:14
  • $\begingroup$ Hi @whuber. If you have two models with different parameters, it is possible to change the units of a parameter which is in just one of the models. So I don't think it could be the case that the units of the parameters affect the maximum likelihood. I have been too vague above: I am specifically thinking about the maximum likelihood. $\endgroup$ – Josh12 Jul 29 '19 at 17:25
  • 1
    $\begingroup$ If you change the units of a parameter then you must change the units in both models, for otherwise they won't be comparable. But since the likelihood is a probability, you're correct that its maximum will not depend on the units of any parameter: indeed, you can even transform the parameters non-linearly (in a one-to-one way) without affecting the maximum. $\endgroup$ – whuber Jul 29 '19 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.