1
$\begingroup$

I recently came across this Answer by @mpiktas wherein he suggested a transformation of $y_i \rightarrow y_i^{\prime}$

$$y_i^{\prime} = (y_i - \overline{y})^2$$

followed by fitting a model for change in mean of $y_i^{\prime}$ as a way to model a change in variance.

The same idea is also present in some versions of Levene's test (although the more robust versions use medians and absolute deviations).

I was wondering then, is there an authoritative or canonical reference for this approach to modelling variance in regression settings? If it matters, the specific application I have in mind is for analysis of irregular time series, but I plan to do the analysis using GLS or GAMs.

$\endgroup$
2
$\begingroup$

The average of squared deviations or residuals provide a biased estimate of the variance when data are IID. The simple degrees of freedom correction removes the bias. Checking the IID assumption means using what you know about the structure of the residuals to check if the average of the residual varies between groups.

In time-series data or geospatial data, you can use variograms. In clustered data, you can calculate intraclass correlation. For sandwich based standard errors, the empirical variance derivation uses a diagonal matrix of residuals as a plug in estimate for variance under no assumptions about IID. In mixed effects modeling, the EM algorithm is used to iteratively estimate fixed effects, and use those residuals to estimate the covariance components of the residuals under some parametric assumptions.

There's nothing more to the rationale of these method than the fact that these are the values used to calculate confidence intervals and p-values under certain assumptions which are verifiable when the data are correlated.

"Robustness" is a funny concept. I think the Huber-White sandwich based standard errors are the only error estimates that are worthy of being called robust. In my simulations, it's usually contrived and data-driven approaches used to justify modeling absolute deviations, or other estimates of "scale". I find them difficult to interpret and, most importantly, less efficient at estimating what I'm interested in.

$\endgroup$
  • $\begingroup$ I wasn't meaning that I wanted to look at the residuals of a model and compute their variance, which is what your answer seems to be getting at. I and the linked question envisaged fitting a model to the transformed values $y_i^{\prime}$ directly, so that rather than the regression model estimating the conditional mean of $y_i$ it was estimating the conditional variance of $y_i$. Levene's test is essentially ANOVA on the transformed $y_i$ (but variations do different transformations). As for motivation; time series models of variance in continuous time seem hard & not widely implemented :-) $\endgroup$ – Gavin Simpson Jul 23 '15 at 17:34
  • 1
    $\begingroup$ @GavinSimpson I don't understand the $y^\prime$ distinction. It's just a squared deviation. Fitting models to these deviations (or residuals...it doesn't matter) is exactly what is done in variograms, intraclass correlations, and in mixed modeling. Levene's test I think does this using the $x$ values themselves and tests for linear heteroscedasticity. Or you can use a more flexible spline approach. $\endgroup$ – AdamO Jul 23 '15 at 17:40
  • $\begingroup$ @AdmaO OK, I see what you mean now, esp re variograms. In the case of the variogram, as I understand it, that is a global assessment of variance; over all pairs of data separation. It's not a model for the variance conditional on $x$ but on separation in space/time, right? I'm interested in a model that allows the variance to change with time, just as a linear regression would allow the mean to change with time, with time being the $x$ data. Hence I'm looking for a canonical reference for this sort of model. Perhaps there isn't one and the justification derives simply from the math? $\endgroup$ – Gavin Simpson Jul 23 '15 at 17:47
  • $\begingroup$ @GavinSimpson Variograms work for both space, space & time, as well as just time. You should be specific... A great model that "allows" for variance to change with time is using robust standard error. This eschews any need to actually model the variance. They aren't called "heteroscedasticity consistent" estimators for nothing. $\endgroup$ – AdamO Jul 23 '15 at 18:18
  • $\begingroup$ We're getting a little away from the specifics of the original question but this discussion is very useful, so thanks. Can I do inference on the robust standard errors? In the R implementation in sandwich I can get & use the covariance matrix but then what? In the specific context I want to ask whether the variance changes with time & where it changes in time, the latter of which would be given by the GAM $y_i^{\prime} \sim \beta_0 + f(\mathrm{time}_i)$. $\endgroup$ – Gavin Simpson Jul 23 '15 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.