What are your go to models for assessing as ACCURATELY as possible the slope and intercept of given predictor and predicted random variables? The goal is to use simulated predictors + outputted intercept and slope to get simulated predicted. I currenlty use a linear regression model but I was wondering if anybody uses something different or has come across any papers that compare models. It could be linear or not, just curious. The data is continous r.v
In many regards, this question is asking how to do statistics. Getting accurate estimates and defining what qualifies as accurate is really the heart of this field.
To be brief, ordinary least squares estimation is the usual way people estimate linear regression coefficients. Among its appealing characteristics are that it is the minimum-variance linear unbiased estimator when the Gauss-Markov conditions hold, and it is a maximum likelihood estimator when we make further normality assumptions about the error terms. However, depending on how we define accuracy, for which the technical term in statistics is loss, there are more accurate estimators, especially if we are willing to use biased estimators.
If we drop the normality assumption, minimizing absolute errors can be more accurate than minimizing squared errors (unbiased but nonlinear estimator).
Certain ridge regressions can be more accurate than OLS (biased linear estimator…nonlinear if you tune the penalty term).
James-Stein estimators also introduce bias in hopes of getting a large reduction in variance that makes it worth the bias.
A central idea in all of these is that mean squared error can be decomposed.
$$ MSE(\hat\theta)=(bias(\hat\theta))^2+var(\hat\theta) $$
Thus, introducing some bias that dramatically decreases the variance can yield an estimator with lower mean squared error (higher accuracy).
This is just if we use squared error, though. Nothing forces us to, and if you want lower absolute error, have at it.
Minimizing absolute loss, regularized regression, and James-Stein estimators are just three classics. I mention them not to be complete but to start readers down the rabbit hole.