Linear regression with error dispersion dependent on the independent variable - Cross Validated most recent 30 from stats.stackexchange.com 2019-11-22T16:20:22Z https://stats.stackexchange.com/feeds/question/393781 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/393781 3 Linear regression with error dispersion dependent on the independent variable Hans https://stats.stackexchange.com/users/44368 2019-02-22T02:36:50Z 2019-04-14T22:06:56Z <p>Suppose <span class="math-container">$y=ax+z$</span> where <span class="math-container">$x, y, z$</span> are random variables with range in <span class="math-container">$\mathbf R$</span>, <span class="math-container">$\mathbf E[x]=0$</span>, the probability distribution <span class="math-container">$p(z|x)$</span> is </p> <p>1) normal distribution <span class="math-container">$N(0,\sigma(x)^2)$</span> with mean <span class="math-container">$0$</span> and standard deviation <span class="math-container">$\sigma(x)$</span> as an unknown function of <span class="math-container">$x$</span>;</p> <p>2) student t-distribution <span class="math-container">$t_{\nu(x)}$</span> with degrees of freedom <span class="math-container">$\nu(x)$</span> an unknown function of <span class="math-container">$x$</span>,</p> <p>and <span class="math-container">$a$</span> is an unknown constant. Suppose <span class="math-container">$(x_i,y_i)_{i=1}^n$</span> is a set of tuples of sample observation of <span class="math-container">$(x,y)$</span>. How do we estimate the following functions?</p> <p>1) <span class="math-container">$(a,\sigma(x))$</span>; </p> <p>2) <span class="math-container">$(a,\nu(x))$</span>.</p> <hr> <p>Note: This is not the heteroscedasticity problem in the conventional sense where the dispersion parameter depends on the index <span class="math-container">$i$</span>. The dispersion parameter now depends on the independent variable <span class="math-container">$x$</span>.</p> https://stats.stackexchange.com/questions/393781/-/393794#393794 3 Answer by user3658307 for Linear regression with error dispersion dependent on the independent variable user3658307 https://stats.stackexchange.com/users/128284 2019-02-22T05:10:46Z 2019-02-22T05:10:46Z <p>My guess is you can reasonably estimate <span class="math-container">$a$</span> with OLS in (1) and a maybe a more robust estimation in (2) like <a href="https://stats.stackexchange.com/questions/236676/can-you-give-a-simple-intuitive-explanation-of-irls-method-to-find-the-mle-of-a/237384#237384">like IRLS</a> (though <a href="https://stats.stackexchange.com/questions/152674/why-is-the-normality-of-residuals-barely-important-at-all-for-the-purpose-of-e">maybe OLS might still be ok</a>). </p> <p>Estimating <span class="math-container">$\sigma(x)$</span> and <span class="math-container">$\nu(x)$</span> is harder. I think you need to decide how to parameterize or quantize wrt <span class="math-container">$x$</span>. For instance, if you choose some function with parameters <span class="math-container">$\theta$</span>, and define your estimate to be <span class="math-container">$\hat{\sigma}(x)=f(x;\theta)$</span>, then you can numerically fit <span class="math-container">$\theta$</span> by maximizing the log-likelihood over the dataset <span class="math-container">$(x_i, z_i)=(x_i,y_i-ax_i)$</span>. The choice of <span class="math-container">$f$</span> represents some level of "prior" over <span class="math-container">$\sigma(x)$</span> or <span class="math-container">$\nu(x)$</span>.</p> <p>I guess you can also <em>jointly</em> estimate <span class="math-container">$a$</span> and <span class="math-container">$\sigma$</span> or <span class="math-container">$\nu$</span> together by combining the two optimization approaches above in an alternating manner.</p> <p>Related Links</p> <ul> <li><p><a href="https://stats.stackexchange.com/questions/350550/mle-for-linear-regression-student-t-distributed-error">MLE for <span class="math-container">$t$</span>-distributed errors</a></p></li> <li><p><a href="https://stats.stackexchange.com/questions/63647/estimating-parameters-of-students-t-distribution">Estimating parameters of <span class="math-container">$t$</span> distributions</a></p></li> <li><p><a href="https://stats.stackexchange.com/questions/29731/regression-when-the-ols-residuals-are-not-normally-distributed">Non-normal OLS residuals</a> (also <a href="https://stats.stackexchange.com/questions/173621/linear-regression-any-non-normal-distribution-giving-identity-of-ols-and-mle">this one</a>)</p></li> </ul>