In my current project I'm working with a previously developed linear model and asking how many training observations do I need to reproduce the model. The original model was developed with ~1000 observations, and I want to see how many I actually need to reproduce the original model parameters. To address this, I plan to sample the training data (keeping the response distribution representative of the training set) and develop new models for numerous repetitions - perhaps 100-1000 repeats. Each time the model is trained, I will use the same set of covariates used in the original model, and I would like to see how the coefficient estimate and standard error compare to the original, but, I'm unclear on how to combine the coefficients and standard errors from 1000 different models. The value for the coefficient could simply be the average or all trials, but how do I combine the standard errors for those same trials?
Let me clarify my thoughts on "reproduce the model parameters:"
What I'm trying to do is to determine how many observations I actually need from the training set to get the same, or close to the same, coefficients. The original model used 937 observations, but I would like to reduce that number. I realize that if I change the number of observations, that will lead to a slightly different set of coefficients with different standard errors from the original full training. The actual value of the coefficients and their standard errors will vary depending on the sample I take from my set of observations. In order to get a handle on the uncertainty, I will repeat my data sampling many times and aggregate the coefficients and their standard errors in order to compare them to the original.
For example, let's say I start with 100 observations. I'll pick 100 observations using a stratified sampling of the original data, train a new model using just those observations, and I'll have a different set of coefficients and standard errors. If I then repeat that process a second time, my coefficients and standard errors will be different due to the different subsample of observations. If I repeat that process 1000 times, I would like to aggregate the 1000 sets of coefficients and standard errors to see how well using 100 observations approximates the coefficients and standard errors I get when using the full dataset.
The question is, how do I combine 1000 sets of coefficients and standard errors in a meaningful way?