Suppose I have a linear model y ~ Xb, and I split my observations into multiple X's X1, X2, X3 etc. What is the most appropriate way to aggregate the separate models y1 ~ X1b1, y2 ~ X2b2 to produce the best estimate of the model that would result from fitting y~ Xb? In other words, I would like to combine all of the parameter vectors b1, b2, b3 etc. into an estimate of b. So far, I have been using an average of the bi's from the separate models weighted by the inverse of the squares of the standard errors of the b values, and it works ok, but I was wondering if there is a formally appropriate way to do it.

  • $\begingroup$ If you can specify the dimensions of each matrix/vector, it will be easy to understand your question. $\endgroup$ – user158565 Jul 31 at 1:13
  • $\begingroup$ Sure, let's say X_full has 1000 observations and 10 features, and the model fit from the entire dataset is y_full ~ X_full*b_full. Next, let's say we split X_full and y_full into batches of 100 each, so we have X_1, X_2, X_3...X_10 and y_1, y_2, y_3...y_10. Suppose that we fit separately 10 models using the 10 batches, giving us 10 estimates of b (b_1, b_2, b_3...b_10). My question is, what is the best way to use the estimates b_1, b_2, b_3...b_10 to estimate b_full. $\endgroup$ – LRMB33 Jul 31 at 13:34
  • $\begingroup$ Then I think it is impossible to find a function of b_1, ..., b_10, such that f(b_1,...,b_10) = b $\endgroup$ – user158565 Jul 31 at 15:19
  • $\begingroup$ I agree, it is impossible to find such an exact function, but I am curious what the best estimate would be, basically I would like to find the function f that minimizes ||f(b_1...b_10) - b_full||2^2 $\endgroup$ – LRMB33 Aug 1 at 15:18
  • $\begingroup$ What is reason that you split the data? You can get b from whole data without any errors $\endgroup$ – user158565 Aug 1 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.