I am currently building a statistical model. According to data, the variance is non-constant, and it is likely based on a factor. Is there any statistical model with variance as a function of other parameters? What is its name, and can that model be fitted in R?

  • $\begingroup$ Your question is so vague that it is unanswerable in its present form. $\endgroup$ – Dilip Sarwate Jun 10 '14 at 16:38
  • $\begingroup$ Here is one example, Variance function regressions for studying inequality (Western & Bloome, 2009). Here is a Pre-print PDF. This is sort of built in already to generalized linear models, as the expected value and the variance are related (e.g. in Poisson model they are equal). $\endgroup$ – Andy W Jun 10 '14 at 16:48
  • 1
    $\begingroup$ GLMs (generalized linear models) can do this, if the variance can be written as a function of the mean. $\endgroup$ – Glen_b Jun 10 '14 at 20:42

This looks like a standard heteroskedastic model, where we treat heteroskedasticity the "old-fashioned way", i.e. by explicitly modelling the error variance as a function of some other variables (which may be the regressors themselves or not). In its most simple form the model is Weighted Least Squares.

Various specifications have been examined in the literature, like for example

$$\sigma^2_i = (\mathbf z_i'\alpha)^2,\;\;\;\sigma^2_i = \exp\{\mathbf z_i'\alpha\}, \;\;\;\sigma^2_i = \sigma^2(\mathbf x_i'\beta)^2$$

The last case indicates that the variance is directly proportional to the conditional expected value of the dependent variable, while in the previous formulations, the $\mathbf z$ vector may contain the regressors or other variables.

The model can be estimated by a two-step least square procedure, or by maximum likelihood -note that the unknown parameter $\alpha$ is common for all $i$ and so we do not have an "incidental parameters" problem.

This approach always had the issue of misspecification that is almost certain to occur in specifying the functional form that characterizes the heteroskedasticity. After the arrival of White's standard errors and the "heteroskedasticity-robust" variance-covariance matrix, the main problem with using OLS regression was cleared away, and the "direct-modeling" approach is visibly less used than in the past, at least in Econometrics.

  • $\begingroup$ This answer is very helpful. Do R have the functionality to solve the problem, say variance is proportional to the factor which is not regressor? $\endgroup$ – ucanuup Jun 11 '14 at 23:05
  • $\begingroup$ I am afraid I cannot help you with R-related questions. But I guess it should be easy to find out. $\endgroup$ – Alecos Papadopoulos Jun 11 '14 at 23:15

To me the question speaks of straight-up mixed models where the typical homogeneous (homoscedastic) error term is (possibly) decomposed into levels and (possibly) explained at each level using functions.

For example, suppose you have a model that looks like this:

(1) $y_{i} = \beta_{0} + \beta_{1}x_{1} + \beta_{2}x_{2} + \varepsilon_{i},$

where, say, $x_{1}$ is some continuous predictor, and $x_{2}$ is a nominal factor (for sake of simplicity).

Now suppose, as you suggest that the variance of $y$ depends on $x_{2}$. You might revise your model as in (2):

(2) $y_{i} = \beta_{0} + \beta_{1}x_{1} + \beta_{2}x_{2} + \varepsilon_{i0} + \varepsilon_{i2},$


$\left[\begin{array}{c}\varepsilon_{0,i}\\ \varepsilon_{2,i}\end{array}\right] \sim \mathcal{N}\left(0,\Omega_{\varepsilon}\right):\Omega_{\varepsilon}=\left[\begin{array}{cc}\sigma^{2}_{\varepsilon 0} & \\ 0 & \sigma^{2}_{\varepsilon 2} \end{array}\right]$

(The covariance in $\Omega_{\varepsilon}$ is assumed zero here, since you've got factors with what I am assuming are mutually exclusive categories. If you are not comfortable with that assumption, and think you can estimate a covariance term, $\sigma_{\varepsilon02}$, go ahead and include that.)

You can even use this model if there is no fixed effect of $x_{2}$ on $y$ (i.e. if $x_{2}$ only contributes a random effect). This could be accomplished implicitly by letting a near zero estimate of $\beta_{2}$ be part of the model as in (2), or explicit as in (3):

(3) $y_{i} = \beta_{0} + \beta_{1}x_{1} + \varepsilon_{0,i} + \varepsilon_{2,i}.$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.