2
$\begingroup$

I do experiments with a certain parameter x. The result is y. I assume y is linearly related to x.

Suppose I can do 1000 experiments, which method will give me a better estimation of the linear relation?

  • Select 1000 different values of x, get a single y for each x, and do linear regression?
  • Select 100 different values of x, run 10 experiments for each x, average the y values for each x, and then do linear regression on the 100 averages?
  • Select 100 different values of x, run 10 experiments for each x, and do linear regression without averaging first?

What if I am not sure that the relation is linear?

$\endgroup$
6
  • $\begingroup$ What do you mean by "better estimation"? Do you mean higher $R^2$ or something else? $\endgroup$
    – Peter Flom
    Nov 9 '12 at 11:37
  • $\begingroup$ I mean that the values I get from the regression (a,b) are closer to the real values. $\endgroup$ Nov 9 '12 at 13:41
  • $\begingroup$ Then what do you mean by "real values"? The values you get from the regression are the real values. $\endgroup$
    – Peter Flom
    Nov 9 '12 at 16:32
  • $\begingroup$ @PeterFlom What do you mean by "The values you get from the regression are the real values"? Aren't the coefficient values you get from the regression just estimates of the real values? $\endgroup$
    – mark999
    Nov 9 '12 at 18:53
  • 1
    $\begingroup$ @PeterFlom Yes I assume that's what Erel wants, together with any other advantages/disadvantages of those methods. $\endgroup$
    – mark999
    Nov 9 '12 at 19:57
3
$\begingroup$

I assume the question refers to the error on the parameter estimates. To assess the linear relationship between two variables x and y we use linear regression to estimate the two parameters intercept and slope.

It is easy to demonstrate that the last two options are identical, because during linear regression we minimize the sum of squared residuals which in this particular case amounts to the same as averaging all y at a particular x value.

However, there is a slight difference between the first option and the last two. We have the same number of data points or measurements, but in the first case we sample y at more x locations; in the second case we sample at any particular x value more often.

The following R code simulates the first and second scenario.

for (i in 1:1e3) {
#1000 different values of x, get a single y for each x
x<-runif(1e3);
noise<-rnorm(length(x),sd=0.1);
y<-x+noise;
p1<-rbind(p1,as.array(lm(y~x)$coeff));

#100 different values of x, run 10 experiments for each x
x_rep<-rep(runif(1e2),times=1e1);
noise<-rnorm(length(x_rep),sd=0.1);
y_rep<-x_rep+noise;
p2<-rbind(p2,as.array(lm(y_rep~x_rep)$coeff));
}; 

# differences between standard deviations of intercepts and slopes
apply(p1,2,sd)[[1]]-apply(p2,2,sd)[[1]]
apply(p1,2,sd)[[2]]-apply(p2,2,sd)[[2]]

The for loop repeats the two scenarios many times, running linear regression each time. At the end we calculate the standard deviation of intercept and slope across repeats. The first scenario might be slightly better as its standard deviation seems to be consistently smaller.

$\endgroup$
1
$\begingroup$

I am not a very good statistician, but since there are no other answers, I will do my best.

  • Solution number 2 is right out, because you are loosing information on the variability -- you have fewer degrees of freedom, and your calculated confidence intervals will be too pessimistic and your tests much less powerful.
  • Solutions number 1 and 3 seem roughly equivalent to me. However, by gut feeling I would prefer (1), especially if you do not know whether the relation is linear.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.