Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I do experiments with a certain parameter x. The result is y. I assume y is linearly related to x.

Suppose I can do 1000 experiments, which method will give me a better estimation of the linear relation?

  • Select 1000 different values of x, get a single y for each x, and do linear regression?
  • Select 100 different values of x, run 10 experiments for each x, average the y values for each x, and then do linear regression on the 100 averages?
  • Select 100 different values of x, run 10 experiments for each x, and do linear regression without averaging first?

What if I am not sure that the relation is linear?

share|improve this question
    
What do you mean by "better estimation"? Do you mean higher $R^2$ or something else? –  Peter Flom Nov 9 '12 at 11:37
    
I mean that the values I get from the regression (a,b) are closer to the real values. –  Erel Segal Halevi Nov 9 '12 at 13:41
    
Then what do you mean by "real values"? The values you get from the regression are the real values. –  Peter Flom Nov 9 '12 at 16:32
    
@PeterFlom What do you mean by "The values you get from the regression are the real values"? Aren't the coefficient values you get from the regression just estimates of the real values? –  mark999 Nov 9 '12 at 18:53
1  
@PeterFlom Yes I assume that's what Erel wants, together with any other advantages/disadvantages of those methods. –  mark999 Nov 9 '12 at 19:57
show 1 more comment

2 Answers

I assume the question refers to the error on the parameter estimates. To assess the linear relationship between two variables x and y we use linear regression to estimate the two parameters intercept and slope.

It is easy to demonstrate that the last two options are identical, because during linear regression we minimize the sum of squared residuals which in this particular case amounts to the same as averaging all y at a particular x value.

However, there is a slight difference between the first option and the last two. We have the same number of data points or measurements, but in the first case we sample y at more x locations; in the second case we sample at any particular x value more often.

The following R code simulates the first and second scenario.

for (i in 1:1e3) {
#1000 different values of x, get a single y for each x
x<-runif(1e3);
noise<-rnorm(length(x),sd=0.1);
y<-x+noise;
p1<-rbind(p1,as.array(lm(y~x)$coeff));

#100 different values of x, run 10 experiments for each x
x_rep<-rep(runif(1e2),times=1e1);
noise<-rnorm(length(x_rep),sd=0.1);
y_rep<-x_rep+noise;
p2<-rbind(p2,as.array(lm(y_rep~x_rep)$coeff));
}; 

# differences between standard deviations of intercepts and slopes
apply(p1,2,sd)[[1]]-apply(p2,2,sd)[[1]]
apply(p1,2,sd)[[2]]-apply(p2,2,sd)[[2]]

The for loop repeats the two scenarios many times, running linear regression each time. At the end we calculate the standard deviation of intercept and slope across repeats. The first scenario might be slightly better as its standard deviation seems to be consistently smaller.

share|improve this answer
add comment

I am not a very good statistician, but since there are no other answers, I will do my best.

  • Solution number 2 is right out, because you are loosing information on the variability -- you have fewer degrees of freedom, and your calculated confidence intervals will be too pessimistic and your tests much less powerful.
  • Solutions number 1 and 3 seem roughly equivalent to me. However, by gut feeling I would prefer (1), especially if you do not know whether the relation is linear.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.