How to match the distribution of one dataset when subsampling from another? I have two datasets that measure the same variable under different conditions. The first dataset (d1) is much larger (9,000 observations) than the second dataset (d2; 900 observations).
A histogram of the two indicates that the distributions between them differ:

Given the disparity between them, I would like to create a sub-sample of d1 to match the distribution of d2. The reason being is that I would like to perform a number of other tests on the data, and would essentially like to 'hold' for this particular variable. Following a previous question that has shown how this could be done in R, i have managed to create a subset (d1a) that seems broadly similar:

What I was hoping to do was to provide some statistical backing that the distributions of d1a and d2 are the same/similar, as indicated by the histogram. I had assumed that a two-sample Kolmogorov-Smirnov test would suffice, but results suggest that the two distributions remain distinct (highly sig. p value).
My questions therefore, are as follows:

*

*is my approach a sensible one to take, given my objectives?

*is my interpretation of the second histogram appropriate (that d1a and d2 are the same/similar)?

*is KS appropriate for larger datasets, or is there an alternative?

*is statistical backing overkill, given that I can point at the histogram and argue that they are at least similar?

 A: Comment continued.  My goal here is not to solve your problem explicitly;  I don't have your
actual data. However, I do want to show that there is no difficulty doing a test
to compare two samples of vastly different sizes and getting reasonable results.
I use fictitious data simulated in R.
Data and summary: I have a large sample x1 of size $n_1 = 9000$
and a small sample x2 of size $n_2 = 900.$ On the whole, my larger sample
tends to have larger values: x1 has a larger sample mean and a larger sample
median than x2.
set.seed(2021)
x1 = rgamma(9000, 3, .01)
x2 = rgamma(900, 1.5, .006)
summary(x1)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
   9.558  174.137  266.524  301.329  391.685 1546.258 
summary(x2)
    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
   1.113  107.969  196.355  262.104  348.345 2072.673 

Two-sample Wilcoxon test. Boxplots (left panel of figure below) show both samples are strongly skewed, sample medians are shown. [The smaller sample has a narrower boxplot.] So a two-sample t test may not be a good idea--even for these very large sample sizes.
However, a nonparametric two-sample Wilcoxon rank sum test shows that a highly significant difference (P-value near 0.)
wilcox.test(x1,x2)

        Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 4911600, p-value < 2.2e-16
alternative hypothesis: true location shift is not equal to 0

Because the two samples have somewhat different shapes, it may not be
appropriate to view this Wilcoxon test as a straightforward comparison of
sample medians. However, empirical CDF (ECDF) plots show that the larger
sample 'dominates' (i.e. mostly has larger values). Thus ECDF plot of the
of the larger sample (blue line in the right panel) is shifted to the right
of the ECDF of the smaller sample (and plots below)

par(mfrow=c(1,2))
 boxplot(x1, x2, col=c("blue","brown"), varwidth=T)
 plot(ecdf(x1), col="blue")
  lines(ecdf(x2), col="brown")
par(mfrow=c(1,1))

Two-sample permutation test. My reticence to do a t test above is because
I can't be sure that a t statistic will have Student's t distribution, so I
wonder if the P-value of a t test will be correct. But it seems that a Welch
2-sample t statistic is a reasonable way to express the difference between the
two samples. So I use the Welch t statistic as the 'metric' in a two-sample
permutation test.
By repeatedly permuting the $n_1+n_2 = 9900$ observations at random (9000 to the first sample
and 900 to the second) and finding the Welch t statistic for each permutation, I can get a good approximation of the permutation
distribution of the metric, and thus a trustworthy approximate P-value.
[Without extra parameters, the R procedure sample randomly scrambles the
elements of its main argument.]
set.seed(415)
x = c(x1,x2)
g = c(rep(1,9000),rep(2,900))
t.obs = t.test(x ~ g)$stat;  t.obs
       t 
5.012354 

t.prm = replicate(1000, t.test(x~sample(g))$stat)
mean(abs(t.prm) >= abs(t.obs))
[1] 0   # P-value of simulated permutation test

hdr = "Permutation Dist'n of Welch T Stat"
hist(t.prm, prob=T, br=30, xlim= c(-6,6), col="skyblue2", main=hdr)
 abline(v = c(t.obs,-t.obs), col="red", lwd=2)


Notes: (1) A Kolmogorov-Smirnov test shows a difference between my two fictitious
samples. (2) More generally, it is not 'overkill' to do an appropriate test--especially not, if there happens to be a significant difference.
ks.test(x1,x2)

        Two-sample Kolmogorov-Smirnov test

data:  x1 and x2
D = 0.19311, p-value < 2.2e-16
alternative hypothesis: two-sided

(3) The K-S test statistic $D$ is the maximum vertical difference between
the two ECDF plots, see figure above.
A: This is a standard matching problem equivalent to the problem of selecting a comparable comparison group from a pool of potential controls when assessing the effect of a treatment in an observational study. The goal of matching is to select a subset of the control pool to resemble the treated units. Instead of sampling, you should use matching. 1:1 optimal matching will work well in this case.
The distributions look somewhat similar, but for some values, there are significant imbalances. Given that it would be straightforward to get an even closer distributional match than the none you got by using optimal matching rather than subsampling, you should try optimal matching before proceeding.
The standard KS test is acceptable for determining whether the two samples come from different distributions. But that is not your question. Your question is whether the sample distributions are similar to each other, similar enough that an analysis performed in one can represent, in some way, the analysis performed in the other. This is a matter of "covariate balance," a topic that has been described at length in the matching literature. There is near-unanimous agreement that hypothesis tests should not be used for assessing balance.
Unfortunately, the question of "Are my distributions similar enough?" is inherently unanswerable without knowing the true values you are trying to estimate. A visual inspection of the histograms (or kernel density plots or ECDFs) is your best bet. With optimal matching, it is likely you will get near-exact distributional balance, except perhaps in the upper tail where there is less support for the values of the target distribution in the comparison pool.
