# Combining coefficients and standard errors from repeated models

In my current project I'm working with a previously developed linear model and asking how many training observations do I need to reproduce the model. The original model was developed with ~1000 observations, and I want to see how many I actually need to reproduce the original model parameters. To address this, I plan to sample the training data (keeping the response distribution representative of the training set) and develop new models for numerous repetitions - perhaps 100-1000 repeats. Each time the model is trained, I will use the same set of covariates used in the original model, and I would like to see how the coefficient estimate and standard error compare to the original, but, I'm unclear on how to combine the coefficients and standard errors from 1000 different models. The value for the coefficient could simply be the average or all trials, but how do I combine the standard errors for those same trials?

Let me clarify my thoughts on "reproduce the model parameters:"

What I'm trying to do is to determine how many observations I actually need from the training set to get the same, or close to the same, coefficients. The original model used 937 observations, but I would like to reduce that number. I realize that if I change the number of observations, that will lead to a slightly different set of coefficients with different standard errors from the original full training. The actual value of the coefficients and their standard errors will vary depending on the sample I take from my set of observations. In order to get a handle on the uncertainty, I will repeat my data sampling many times and aggregate the coefficients and their standard errors in order to compare them to the original.

For example, let's say I start with 100 observations. I'll pick 100 observations using a stratified sampling of the original data, train a new model using just those observations, and I'll have a different set of coefficients and standard errors. If I then repeat that process a second time, my coefficients and standard errors will be different due to the different subsample of observations. If I repeat that process 1000 times, I would like to aggregate the 1000 sets of coefficients and standard errors to see how well using 100 observations approximates the coefficients and standard errors I get when using the full dataset.

The question is, how do I combine 1000 sets of coefficients and standard errors in a meaningful way?

• Although what you mean by "reproduce the original model parameters" isn't completely clear, what you describe seems to be something like the well established techniques of cross-validation or the bootstrap, which are tags on this site. This page and the pages linked from it should be a good place to start.
– EdM
Commented May 19, 2015 at 19:47
• Thanks for the comment. I added some extra text to try and explain what I'm trying to do. Commented May 19, 2015 at 20:42
• What is to be gained from reducing the number of observations, if you already have all 937 observations? For what purpose will you use your model?
– EdM
Commented May 19, 2015 at 20:58
• The purpose is to reduce costs. We have to re-evaluate the observations/samples and to do so with all 937 will be cost prohibitive. Commented May 19, 2015 at 21:05

If you are keeping the same set of covariates in each of your new models based on subsets of the original observations, the fact that you have a linear model means that your work is essentially already done. Each coefficient standard error reported in your model with all 937 cases essentially contains a factor close to inversely proportional to the square root of the number of cases. If you analyze subsets with 100 cases each, your coefficients in the subset models will have about $\sqrt{\frac{937}{100}}$ higher standard errors, about a factor of 3.