I'm working on a Synthetic Data Generation model, and I'm confused about a metric mentioned in a seminal paper (details of paper added below)
Context: Synthetic Data Generation involves sampling from the population, pushing the sample through an algorithm, and generating a dataset. Multiple synthesis (say 1000 times, all initialized with different weights for the algorithm) generate 1000 different synthesized datasets. A metric for measuring the analytical validity of the datasets are required, and the authors use Coverage probability at 95% confidence intervals of a fixed number of point estimands (linear regression coefficients, logistic regression coefficients, etc) generated using a given synthesized dataset.
How do you calculate the coverage probability of an estimand (of say, a regression done) from a dataset?
What I know: I understand calculating the coverage probability of a population property - say mean - which involves 1) Drawing a sample, calculating the mean of the sample, computing a 95% Confidence Interval of the mean for this sample, 2) Repeating (1) for all samples, and (3) Calculating how many of all the Sample Confidence Intervals actually contained the true population mean, giving you the coverage rate.
How do I extrapolate this to regression coefficients? I can generate a sample synthetic dataset, use that dataset for a particular regression and obtain all the estimands. How do you calculate the confidence interval of this estimand? For mean, (assuming normal distribution) I could directly use the t statistic with significance α and n-1 degrees of freedom to formulate the confidence interval (explained in more detail here), but is that relevant for the regression co-efficients too?
Details: Paper link | Drechsler, Jörg, and Jerome P. Reiter. "An empirical evaluation of easily implemented, nonparametric methods for generating synthetic datasets." Computational Statistics & Data Analysis 55.12 (2011): 3232-3243. Relevant part: Section 4.2 - Analytical Validity