3 added 1737 characters in body edited May 21 at 6:05 Lovecraft 2122 bronze badges 2 added 1737 characters in body edited May 20 at 20:22 Lovecraft 2122 bronze badges I really felt i needneeded to clarify this question even though it is fairly old, sincebecause there isare a lot of misconceptions in herethis thread. The MLE (Maximum likelihood estimator) & QMLE (Quasi Maximum Likelihood Estimator) of the ARCH(pq) model is both consistent and asymptotically normal for specific parameter settingsranges of the parameters depending on the levelsize of pq. There are less parameters than data points provided that the available dataset is Larger than the amount of parameters sought estimating. This is very rarely a problem, since the prime use of the ARCH model is financial time series, in which there are often several thousand data points. So the amount of degrees of freedom used for estimation will rarely be a problem. The laglength, q, is NOT defined by the amount of data. Further it is assumed that $$z_{t}$$ is independent across all t, and that $$\alpha_{1} \geq 0$$ and $$\omega>0$$. The two last assumptions ensure that $$\sigma_{t}^2$$ is always positive. The Classical ARCH model only opperatesoperates with equation (2) and (3), equation (1) is altered a bit such that we may look at $$\epsilon_{t}$$ as the demeaned return of some stock. IntroducingThe reason for this specification can be thought of as the assumption that stoch markeds are atleast weakly efficient, which tends to be true in empirical studies. Alleviating the assumption of weak efficiency one could introducing p lags ogof $$y_{t}$$ intoto equation (1). This will turn itthe model into the AR(p)-ARCH(1) model, which is a whole other model with different requirements for stability and thus estimator properties - so i will omit these as they are much more complicated to dealwork with (although this is rarely mentioned in introductory ARCH/GARCH courses). The first thing to clarify is that equation (2) is by all means stochastic andwhich verifies that the model does have a residual. $$\epsilon_{t}$$ is not known given the information set $$I_{t-1}=\{y_{0},y_{1},...,y_{t-1}\}$$ however the function for the conditional variance, $$\sigma_{t}^2$$ is. This is what they mean by the fact that $$\sigma_{t}^2$$ is deterministic, or rather constant conditional on $$I_{t-1}$$ in opposition to the stochastic volatility models (see for example the log normal stochastic volatility model). In order to make sense of this model, and how to estimate it by MLE we have to know the conditional distribution for $$y_{t}$$ given $$I_{t-1}$$, such that we can write the conditional likelihood function. If $$z_{t}$$ is not normal but maybie a more heavy tailed process we may still, in some cases, think of it as normally distributed during estimation to obtain a well behaved QMLE. To obtain the conditional distribution If we takefirst derive the conditonal mean and variance of $$y_{t}$$. We start by the conditional expectationmean of (1) first: By abusing normality of $$z_{t}$$, namely the rule that for any normal distributed variable $$q\sim N(m,v)$$ and constants $$(a,b)\in\mathbb{R}^{2}$$ it holds that the linear combination given by $$l=a+bq$$ follows a normal distribution given by $$N(bv+a,vb^{2})$$$$N(bm+a,vb^{2})$$. Where the number T is the total number of observations excluding the initalinitial one, which we condition on, and $$\theta=(\delta,\omega,\alpha_{1})$$ is our parameter vector we seek to estimate. The loglikelihoodlog-likelihood function is given by: By definition of the likelihood problem we have 3 equations given by the First order conditions (FOC) and 3 parameters in $$\theta$$. Because of this and the functional form of equations (1), (2) and (3) each of the estimates are identified by the FOC's. So there is no problems with estimation arising from this either. To close the final bit of confusion that seems to have appeared here. We run into the problem that the FOC repressentingrepresenting the maximization problem does not have an analytical solution. That is there is no way to solve the equation system by standard math - or rather no formula. This does not mean that no solution exists. That is by using numerical optimization such as BFGS, Newton-Rahpson or Nelder mead one can obtain a solution to the 3x3 equation system represented by this particular problem. For this particular problem, the MLE estimator is consistent and asymptotically normal for the estimator of $$\alpha_{1}$$ and $$\omega$$ as long as $$y_{t}$$ is strictly stationary. This holds when $$\alpha_{1}$$ is smaller than approximately 3.56. See Daniel B. Nelson (1990) "STATIONARITY AND PERSISTENCE IN THE GARCH(1,1) MODEL" Econometric theory, 6, nr 3, 318-334 as well as Jensen & Rahbek (2004) "ASYMPTOTIC NORMALITY OF THE QMLE ESTIMATOR OF ARCH IN THE NONSTATIONARY CASE" Econometrica, 72, nr. 2, 641-646. Further the last paper shows that the MLE estimator of $$\alpha_{1}$$ is consistent and asymptotically normal for all values of $$\alpha_{1}$$ which is a very strong result. This suggests that verifying the behaviour of the other parameter estimates are simple, since one just need to test of $$\alpha_{1}<3.56$$. This is a property only known to hold for the MLE and QMLE, and is the reason why parameters in ARCH models are almost never estimated with other estimators. I would conjecture however that the estimator of $$\delta$$ is only asymptoticaly normal given that $$\alpha_{1}<1$$, and consistent if $$\alpha_{1}<\pi/2$$. This is because $$\alpha_{1}>1$$ ensures that $$E(y_{t}^2)=\infty$$ mich messes up the CLT, and $$\alpha_{1}>\pi/2$$ ensures that $$E(y_{t})$$ is undefined - messing up the LLN for ergodic and stationary series. However, this might not cause problems, as it does not do so for estimation of $$\alpha_{1}$$ and $$\omega$$. To verify this, one can analyze when the FOC of the MLE satisfies lemma 2 in Jensen & Rahbek (2004b) (Asymptotic inference for nonstationary GARCH.). This is a slightly more complicated endevour, and i will leave that to the reader to verify this if needed be. I really felt i need to clarify this question even though it is fairly old, since there is a lot of misconceptions in here. The MLE (Maximum likelihood estimator) & QMLE (Quasi Maximum Likelihood Estimator) of the ARCH(p) model is both consistent and asymptotically normal for specific parameter settings depending on the level of p. There are less parameters than data points provided that the available dataset is Larger than the amount of parameters sought estimating. This is very rarely a problem, since the prime use of the ARCH model is financial time series, in which there are often several thousand data points. Further it is assumed that $$z_{t}$$ is independent across all t. The Classical ARCH model only opperates with equation (2) and (3), equation (1) is altered a bit such that we may look at $$\epsilon_{t}$$ as the demeaned return of some stock. Introducing lags og $$y_{t}$$ into (1) will turn it into the AR(p)-ARCH(1) model, which is a whole other model with different requirements for stability and thus estimator properties - so i will omit these as they are much more complicated to deal with (although this is rarely mentioned in introductory ARCH/GARCH courses). The first thing to clarify is that equation (2) is by all means stochastic and verifies that the model does have a residual. $$\epsilon_{t}$$ is not known given the information set $$I_{t-1}=\{y_{0},y_{1},...,y_{t-1}\}$$ however the function for the conditional variance, $$\sigma_{t}^2$$ is. This is what they mean by the fact that $$\sigma_{t}^2$$ is deterministic, or rather constant conditional on $$I_{t-1}$$ in opposition to the stochastic volatility models (see for example the log normal stochastic volatility model). In order to make sense of this model, and how to estimate it by MLE we have to know the conditional distribution for $$y_{t}$$ given $$I_{t-1}$$, such that we can write the conditional likelihood function. If $$z_{t}$$ is not normal but maybie a more heavy tailed process we may still, in some cases, think of it as normally distributed during estimation to obtain a well behaved QMLE. To obtain the conditional distribution If we take the conditional expectation of (1) first: By abusing normality of $$z_{t}$$, namely that for any normal distributed variable $$q\sim N(m,v)$$ and constants $$(a,b)\in\mathbb{R}^{2}$$ it holds that the linear combination given by $$l=a+bq$$ follows a normal distribution given by $$N(bv+a,vb^{2})$$. Where the number T is the total number of observations excluding the inital one, which we condition on, and $$\theta=(\delta,\omega,\alpha_{1})$$ is our parameter vector we seek to estimate. The loglikelihood function is given by: To close the final bit of confusion that seems to have appeared here. We run into the problem that the FOC repressenting the maximization problem does not have an analytical solution. That is there is no way to solve the equation system by standard math - or rather no formula. This does not mean that no solution exists. That is by using numerical optimization such as BFGS, Newton-Rahpson or Nelder mead one can obtain a solution to the 3x3 equation system represented by this particular problem. For this particular problem, the MLE estimator is consistent and asymptotically normal for the estimator of $$\alpha_{1}$$ and $$\omega$$. See Daniel B. Nelson (1990) "STATIONARITY AND PERSISTENCE IN THE GARCH(1,1) MODEL" Econometric theory, 6, nr 3, 318-334 as well as Jensen & Rahbek (2004) "ASYMPTOTIC NORMALITY OF THE QMLE ESTIMATOR OF ARCH IN THE NONSTATIONARY CASE" Econometrica, 72, nr. 2, 641-646. I would conjecture however that the estimator of $$\delta$$ is only asymptoticaly normal given that $$\alpha_{1}<1$$, and consistent if $$\alpha_{1}<\pi/2$$. This is because $$\alpha_{1}>1$$ ensures that $$E(y_{t}^2)=\infty$$ mich messes up the CLT, and $$\alpha_{1}>\pi/2$$ ensures that $$E(y_{t})$$ is undefined - messing up the LLN for ergodic and stationary series. However, this might not cause problems, as it does not do so for estimation of $$\alpha_{1}$$ and $$\omega$$. I really felt i needed to clarify this question even though it is fairly old, because there are a lot of misconceptions in this thread. The MLE (Maximum likelihood estimator) & QMLE (Quasi Maximum Likelihood Estimator) of the ARCH(q) model is both consistent and asymptotically normal for specific ranges of the parameters depending on the size of q. There are less parameters than data points provided that the available dataset is Larger than the amount of parameters sought estimating. This is very rarely a problem, since the prime use of the ARCH model is financial time series, in which there are often several thousand data points. So the amount of degrees of freedom used for estimation will rarely be a problem. The laglength, q, is NOT defined by the amount of data. Further it is assumed that $$z_{t}$$ is independent across all t, and that $$\alpha_{1} \geq 0$$ and $$\omega>0$$. The two last assumptions ensure that $$\sigma_{t}^2$$ is always positive. The Classical ARCH model only operates with equation (2) and (3), equation (1) is altered a bit such that we may look at $$\epsilon_{t}$$ as the demeaned return of some stock. The reason for this specification can be thought of as the assumption that stoch markeds are atleast weakly efficient, which tends to be true in empirical studies. Alleviating the assumption of weak efficiency one could introducing p lags of $$y_{t}$$ to equation (1). This will turn the model into the AR(p)-ARCH(1) model, which is a whole other model with different requirements for stability and thus estimator properties - so i will omit these as they are much more complicated to work with (although this is rarely mentioned in introductory ARCH/GARCH courses). The first thing to clarify is that equation (2) is by all means stochastic which verifies that the model does have a residual. $$\epsilon_{t}$$ is not known given the information set $$I_{t-1}=\{y_{0},y_{1},...,y_{t-1}\}$$ however the function for the conditional variance, $$\sigma_{t}^2$$ is. This is what they mean by the fact that $$\sigma_{t}^2$$ is deterministic, or rather constant conditional on $$I_{t-1}$$ in opposition to the stochastic volatility models (see for example the log normal stochastic volatility model). In order to make sense of this model, and how to estimate it by MLE we have to know the conditional distribution for $$y_{t}$$ given $$I_{t-1}$$, such that we can write the conditional likelihood function. If $$z_{t}$$ is not normal but a more heavy tailed process we may still, in some cases, think of it as normally distributed during estimation to obtain a well behaved QMLE. To obtain the conditional distribution we first derive the conditonal mean and variance of $$y_{t}$$. We start by the conditional mean of (1) first: By abusing normality of $$z_{t}$$, namely the rule that for any normal distributed variable $$q\sim N(m,v)$$ and constants $$(a,b)\in\mathbb{R}^{2}$$ it holds that the linear combination given by $$l=a+bq$$ follows a normal distribution given by $$N(bm+a,vb^{2})$$. Where the number T is the total number of observations excluding the initial one, which we condition on, and $$\theta=(\delta,\omega,\alpha_{1})$$ is our parameter vector we seek to estimate. The log-likelihood function is given by: By definition of the likelihood problem we have 3 equations given by the First order conditions (FOC) and 3 parameters in $$\theta$$. Because of this and the functional form of equations (1), (2) and (3) each of the estimates are identified by the FOC's. So there is no problems with estimation arising from this either. To close the final bit of confusion that seems to have appeared here. We run into the problem that the FOC representing the maximization problem does not have an analytical solution. That is there is no way to solve the equation system by standard math - or rather no formula. This does not mean that no solution exists. That is by using numerical optimization such as BFGS, Newton-Rahpson or Nelder mead one can obtain a solution to the 3x3 equation system represented by this particular problem. For this particular problem, the MLE estimator is consistent and asymptotically normal for the estimator of $$\alpha_{1}$$ and $$\omega$$ as long as $$y_{t}$$ is strictly stationary. This holds when $$\alpha_{1}$$ is smaller than approximately 3.56. See Daniel B. Nelson (1990) "STATIONARITY AND PERSISTENCE IN THE GARCH(1,1) MODEL" Econometric theory, 6, nr 3, 318-334 as well as Jensen & Rahbek (2004) "ASYMPTOTIC NORMALITY OF THE QMLE ESTIMATOR OF ARCH IN THE NONSTATIONARY CASE" Econometrica, 72, nr. 2, 641-646. Further the last paper shows that the MLE estimator of $$\alpha_{1}$$ is consistent and asymptotically normal for all values of $$\alpha_{1}$$ which is a very strong result. This suggests that verifying the behaviour of the other parameter estimates are simple, since one just need to test of $$\alpha_{1}<3.56$$. This is a property only known to hold for the MLE and QMLE, and is the reason why parameters in ARCH models are almost never estimated with other estimators. I would conjecture however that the estimator of $$\delta$$ is only asymptoticaly normal given that $$\alpha_{1}<1$$, and consistent if $$\alpha_{1}<\pi/2$$. This is because $$\alpha_{1}>1$$ ensures that $$E(y_{t}^2)=\infty$$ mich messes up the CLT, and $$\alpha_{1}>\pi/2$$ ensures that $$E(y_{t})$$ is undefined - messing up the LLN for ergodic and stationary series. However, this might not cause problems, as it does not do so for estimation of $$\alpha_{1}$$ and $$\omega$$. To verify this, one can analyze when the FOC of the MLE satisfies lemma 2 in Jensen & Rahbek (2004b) (Asymptotic inference for nonstationary GARCH.). This is a slightly more complicated endevour, and i will leave that to the reader to verify this if needed be. 1 answered May 20 at 15:29 Lovecraft 2122 bronze badges I really felt i need to clarify this question even though it is fairly old, since there is a lot of misconceptions in here. The MLE (Maximum likelihood estimator) & QMLE (Quasi Maximum Likelihood Estimator) of the ARCH(p) model is both consistent and asymptotically normal for specific parameter settings depending on the level of p. There are less parameters than data points provided that the available dataset is Larger than the amount of parameters sought estimating. This is very rarely a problem, since the prime use of the ARCH model is financial time series, in which there are often several thousand data points. To completely clarify this, ill write up the ARCH(1) model in a similar way to what Robert Engle did in his seminal work on the ARCH model back in 1982. The ARCH model is repressented by the following 3 equations: $$(1)\,\,y_{t}=\delta+\epsilon_{t}$$ $$(2)\,\,\epsilon_{t}=\sigma_{t}z_{t},\,\,z_{t}\sim N(0,1)$$ $$(3)\,\sigma_{t}^{2} =\omega+\alpha\epsilon_{t-1}^{2}$$ Further it is assumed that $$z_{t}$$ is independent across all t. The Classical ARCH model only opperates with equation (2) and (3), equation (1) is altered a bit such that we may look at $$\epsilon_{t}$$ as the demeaned return of some stock. Introducing lags og $$y_{t}$$ into (1) will turn it into the AR(p)-ARCH(1) model, which is a whole other model with different requirements for stability and thus estimator properties - so i will omit these as they are much more complicated to deal with (although this is rarely mentioned in introductory ARCH/GARCH courses). The first thing to clarify is that equation (2) is by all means stochastic and verifies that the model does have a residual. $$\epsilon_{t}$$ is not known given the information set $$I_{t-1}=\{y_{0},y_{1},...,y_{t-1}\}$$ however the function for the conditional variance, $$\sigma_{t}^2$$ is. This is what they mean by the fact that $$\sigma_{t}^2$$ is deterministic, or rather constant conditional on $$I_{t-1}$$ in opposition to the stochastic volatility models (see for example the log normal stochastic volatility model). In order to make sense of this model, and how to estimate it by MLE we have to know the conditional distribution for $$y_{t}$$ given $$I_{t-1}$$, such that we can write the conditional likelihood function. If $$z_{t}$$ is not normal but maybie a more heavy tailed process we may still, in some cases, think of it as normally distributed during estimation to obtain a well behaved QMLE. To obtain the conditional distribution If we take the conditional expectation of (1) first: $$E(y_{t}|I_{t-1})=E(\delta+\epsilon_{t}|I_{t-1})=\delta + E(\sigma_{t}z_{t}|I_{t-1})$$ Because $$\sigma_{t}$$ can be calculated by through $$I_{t-1}$$, and the independence assumption in regard to $$z_{t}$$ implies that $$E(z_{t}|I_{t-1})=E(z_{t})$$ we obtain: $$E(y_{t}|I_{t-1})=\delta+\sigma_{t}E(z_{t})=\delta$$ Thus we have derived the conditional mean. Now we derive the conditional variance: $$Var(y_{t}|I_{t-1})=E((y_{t}-E(y_{t}|I_{t-1}))^2|I_{t-1})=E((y_{t}-\delta)^2|I_{t-1})=E(\epsilon_{t}^2|I_{t-1})$$ We substitute in equation (2) and obtain: $$Var(y_{t}|I_{t-1})=E(\sigma_{t}^2z_{t}^2|I_{t-1})=\sigma_{t}^2E(z_{t}^2|I_{t-1})$$ In the last equality we used that $$\sigma_{t}$$ is parfectly defined by the informationset, and that $$z_{t}$$ is independent of itself across time. Using that $$z_{t}\sim N(0,1)$$ we get $$Var(y_{t}|I_{t-1})=\sigma_{t}^2$$ By abusing normality of $$z_{t}$$, namely that for any normal distributed variable $$q\sim N(m,v)$$ and constants $$(a,b)\in\mathbb{R}^{2}$$ it holds that the linear combination given by $$l=a+bq$$ follows a normal distribution given by $$N(bv+a,vb^{2})$$. We obtain that: $$y_{t}|I_{t-1}\sim N(\delta,\sigma_{t}^2)$$. This gives us the conditional likelihood function given by: $$L(\theta|y_{0})=\prod_{t=1}^{T}\frac{1}{\sqrt{2\pi\sigma_{t}^{2}}}\exp\left\{ -\frac{1}{2}\frac{(y_{t}-\delta)^{2}}{\sigma_{t}^{2}}\right\}$$ Where the number T is the total number of observations excluding the inital one, which we condition on, and $$\theta=(\delta,\omega,\alpha_{1})$$ is our parameter vector we seek to estimate. The loglikelihood function is given by: $$\mathcal{L}(\theta|y_{0})=-\frac{1}{2}\sum_{i=1}^{T}\left[\ln(2\pi)+\ln(\sigma_{t}^{2})+\frac{(y_{t}-\delta)^{2}}{\sigma_{t}^{2}}\right]$$ And finally the maximum likelihood estimator is given by: $$\hat{\theta}_{ML}=\arg\max_{\theta}\mathcal{L}(\theta|y_{0})$$ To close the final bit of confusion that seems to have appeared here. We run into the problem that the FOC repressenting the maximization problem does not have an analytical solution. That is there is no way to solve the equation system by standard math - or rather no formula. This does not mean that no solution exists. That is by using numerical optimization such as BFGS, Newton-Rahpson or Nelder mead one can obtain a solution to the 3x3 equation system represented by this particular problem. For this particular problem, the MLE estimator is consistent and asymptotically normal for the estimator of $$\alpha_{1}$$ and $$\omega$$. See Daniel B. Nelson (1990) "STATIONARITY AND PERSISTENCE IN THE GARCH(1,1) MODEL" Econometric theory, 6, nr 3, 318-334 as well as Jensen & Rahbek (2004) "ASYMPTOTIC NORMALITY OF THE QMLE ESTIMATOR OF ARCH IN THE NONSTATIONARY CASE" Econometrica, 72, nr. 2, 641-646. I would conjecture however that the estimator of $$\delta$$ is only asymptoticaly normal given that $$\alpha_{1}<1$$, and consistent if $$\alpha_{1}<\pi/2$$. This is because $$\alpha_{1}>1$$ ensures that $$E(y_{t}^2)=\infty$$ mich messes up the CLT, and $$\alpha_{1}>\pi/2$$ ensures that $$E(y_{t})$$ is undefined - messing up the LLN for ergodic and stationary series. However, this might not cause problems, as it does not do so for estimation of $$\alpha_{1}$$ and $$\omega$$.