Skip to main content
added 112 characters in body
Source Link
dipetkov
  • 10.7k
  • 2
  • 20
  • 56

The Aikake information criterion (AIC) is derived by minimizing the Kullback–Leibler (KL) divergence between the data-generating distribution $g(y)$ and an approximating model $f_\theta(y)$ with parameters $\theta$. Asymptotically, $\operatorname{KL}(g \| f_\theta)$KL$(g \| f_\theta)$ is minimized by the maximum likelihood estimator $\hat{\theta}_n$.

So we can use AIC to compare two generalized linear models (GLMs) — even if the likelihood functions are different — if we use (a) maximum likelihood and (b) the same data to estimate the model parameters $\theta_1$ and $\theta_2$. The comparison will tell us which model, $f_{\theta_1}$ or $f_{\theta_2}$, is the "closer approximation" for the truth $g$.

The theory behind AIC doesn't make assumptions about the form of the models $f_{\theta_1}$ and $f_{\theta_2}$, except the usual "mild regularity conditions". "Same data" means the same observations (data points) and the same outcome variable for each observation. Basically, the same data-generating distribution $g(y)$. That's why we can't compare a model for $y$ and one for a transformation such as $\log(y)$; a model for a continuous $y$ and one for a discretized version of $y$; two models with different supports for $y$, etc.

The Aikake information criterion (AIC) is derived by minimizing the Kullback–Leibler (KL) divergence between the data-generating distribution $g(y)$ and an approximating model $f_\theta(y)$ with parameters $\theta$. Asymptotically, $\operatorname{KL}(g \| f_\theta)$ is minimized by the maximum likelihood estimator $\hat{\theta}_n$.

So we can use AIC to compare two generalized linear models (GLMs) — even if the likelihood functions are different — if we use (a) maximum likelihood and (b) the same data to estimate the model parameters $\theta_1$ and $\theta_2$. The theory behind AIC doesn't make assumptions about the form of the models $f_{\theta_1}$ and $f_{\theta_2}$, except the usual "mild regularity conditions". "Same data" means the same observations (data points) and the same outcome variable for each observation. Basically, the same data-generating distribution $g(y)$. That's why we can't compare a model for $y$ and one for a transformation such as $\log(y)$; a model for a continuous $y$ and one for a discretized version of $y$; two models with different supports for $y$, etc.

The Aikake information criterion (AIC) is derived by minimizing the Kullback–Leibler (KL) divergence between the data-generating distribution $g(y)$ and an approximating model $f_\theta(y)$ with parameters $\theta$. Asymptotically, KL$(g \| f_\theta)$ is minimized by the maximum likelihood estimator $\hat{\theta}_n$.

So we can use AIC to compare two generalized linear models (GLMs) — even if the likelihood functions are different — if we use (a) maximum likelihood and (b) the same data to estimate the model parameters $\theta_1$ and $\theta_2$. The comparison will tell us which model, $f_{\theta_1}$ or $f_{\theta_2}$, is the "closer approximation" for the truth $g$.

The theory behind AIC doesn't make assumptions about the form of the models $f_{\theta_1}$ and $f_{\theta_2}$, except the usual "mild regularity conditions". "Same data" means the same observations (data points) and the same outcome variable for each observation. Basically, the same data-generating distribution $g(y)$. That's why we can't compare a model for $y$ and one for a transformation such as $\log(y)$; a model for a continuous $y$ and one for a discretized version of $y$; two models with different supports for $y$, etc.

Source Link
dipetkov
  • 10.7k
  • 2
  • 20
  • 56

The Aikake information criterion (AIC) is derived by minimizing the Kullback–Leibler (KL) divergence between the data-generating distribution $g(y)$ and an approximating model $f_\theta(y)$ with parameters $\theta$. Asymptotically, $\operatorname{KL}(g \| f_\theta)$ is minimized by the maximum likelihood estimator $\hat{\theta}_n$.

So we can use AIC to compare two generalized linear models (GLMs) — even if the likelihood functions are different — if we use (a) maximum likelihood and (b) the same data to estimate the model parameters $\theta_1$ and $\theta_2$. The theory behind AIC doesn't make assumptions about the form of the models $f_{\theta_1}$ and $f_{\theta_2}$, except the usual "mild regularity conditions". "Same data" means the same observations (data points) and the same outcome variable for each observation. Basically, the same data-generating distribution $g(y)$. That's why we can't compare a model for $y$ and one for a transformation such as $\log(y)$; a model for a continuous $y$ and one for a discretized version of $y$; two models with different supports for $y$, etc.