2
$\begingroup$

My question seems to be very basic one but my search has not given any similar question. I have small dataset of 8 $(x,y)$ values with uncertainties for $y$ (dependent variable) and the theory predicts quadratic dependence $y=a x^2 + b x + c$. I wish to fit this dataset to the quadratic equation and to calculate confidence intervals for the parameters $a$, $b$, $c$ based both on the residuals (they are very small, i.e. the quadratic model describes the experimental data with very small residuals) and on declared uncertainty for every measurement (the uncertainties are much bigger than the residuals). What is the most correct way to do this?

$\endgroup$
3
$\begingroup$

Based on the OP's response to my questions, I'd suggest boostrapping, which will take into account the possibility that your small sample size results in overfitting the model. I would look into the body of theory called Error in Variables Regression. See link and link

$\endgroup$
  • $\begingroup$ Other, related search terms include Deming regression, Model II regression, total least squares. $\endgroup$ – Glen_b Oct 23 '13 at 22:45
  • $\begingroup$ @Eupraxis1981 Thank you for the links, they give some information but no formulas. According to the Wikipedia link you provided, "errors-in-variables models are regression models that account for measurement errors in the independent variables" (emphasis mine). In my case I assume that the independent variable x has no error and the error of dependent variable y is known. Your first link contains formula only for linear model with error in the independent variable. $\endgroup$ – Alexey Popkov Oct 24 '13 at 6:55
  • $\begingroup$ @Alexey: I guess I am having trouble understanding what you mean by uncertainty in y, apart from the residuals. If your model has small residuals, then in general that is good and indicates that the model accounts for most of the uncertainty in Y. Hoever, since you have a small model, you are using up 1/2 of your degrees of freedom by fitting a quadratic model, so there is a potential for bias vs. fit. Is that what you are tryign to account for? $\endgroup$ – user31668 Oct 24 '13 at 12:02
  • 1
    $\begingroup$ You don't. Regression is precisely for when you have uncertain measurement on Y. You will not end up using your assumed uncertainty in Y, it should be reflected in the residuals. If you are concerned that the small residuals is the result of spurious fitting to a small dataset, then you will need to do boostrapping to estimate the contribution from small-sample bias vs. residual error. The CIs from a regression package will not differentiate for you, and it is not a simple calcuation. $\endgroup$ – user31668 Oct 24 '13 at 14:59
  • 1
    $\begingroup$ Yes, you can use weighted least squares: itl.nist.gov/div898/handbook/pmd/section1/pmd143.htm which allows you to enter your specific errors for each measurement. Then you can use the standard errors from the package to form your CIs. $\endgroup$ – user31668 Oct 24 '13 at 15:36
0
$\begingroup$

Let $z_i$ be true value of $y_i$. If $z_i \sim N(y_i, \Delta y_i)$, and the theory predicts $z_i=a x_i^2$ then you should minimize $$\sum_i \left( {a x_i^2 + b x_i + c - y_i \over \Delta y_i} \right)^2$$

This is a weighted linear regression problem.

$\endgroup$
  • $\begingroup$ The theory predicts $z_i=a x_i^2 + b x_i + c$. And I need to compute confidence intervals, especially for the free term $c$. $\endgroup$ – Alexey Popkov Oct 24 '13 at 7:27
  • $\begingroup$ The answer is edited. $\endgroup$ – user31264 Oct 24 '13 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.