# Sampling from dataset according to distribution obtained from another dataset

Suppose we have dataset $$A$$ with several categorical and numerical features: $$A_{cat_1}$$, $$A_{cat_2}$$, $$\ldots;$$ $$A_{num_1}$$, $$A_{num_2}$$, $$\ldots;$$

Also we have another dataset $$B$$ with the same features, but probably with less categories (i.e. unique values) in features.

We want to sample from dataset $$B$$ according to joint distribution of $$A$$. How can one do it?

I was thinking in the following direction:

1. We can LabelEncode categorical features. After that we can use Kernel Density Estimation on both numerical and encoded categorical features. Is it the right way to estimate distribution of $$A$$ or maybe there is more correct procedure to estimate distribution in presence of categorical variables?
2. After we obtain KDE of $$A$$, we need to sample from $$B$$ according to that distribution. Could you describe (or maybe provide some code) how can one do it?

I admit this path itself could not be the best solution, so I am also open to better suggestions or any sources.

So I think if I understand you correctly, what you are trying to do is to sample points from $$B$$ according to the distribution of $$A$$. In which case I would take a different approach to what you have laid out here.

First of all you talked about label encoding $$A$$ and then doing a KDE. I don't think this makes sense unless your categorical variables can really present something continuous ie colour $$\in$$ [red, orange, yellow, green] as opposed to pet $$\in$$ [cat, frog, horse]. The problem here is that if you label encode these and apply the KDE filter, unless you set a small width for the filter you will end up with a lot of probability distribution between theses discrete categories. You will also end up blurring your distributions for the other variables.

I think a better approach would be this:

1. Randomly sample a datapoint $$a$$ from $$A$$
2. Find the datapoint $$b$$ from $$B$$ which minimises some distance $$D(a,b)$$
3. Repeat

Then all you need to do is define a distance function $$D$$. This could take into account both continuous values and the categorical values.

For categorical variables perhaps you decide the partial distance $$d_{cat_1} = 0\textit{ if }(a_{cat_1}=b_{cat_1})\textit{ else } \infty$$

and for continuous variables maybe you could use $$d_{num_1} = (a_{num_1}-a_{num_2})/\sigma_{num_1}$$ where $$\sigma_{num_1}$$ is the standard deviation of $$[A_{num_1}, B_{num_1}]$$ and so on.

Then you could set $$D = \sum_x d_x^2$$

Here is a minimum toy example in python

import matplotlib.pyplot as plt
A = np.random.normal(0,1,(1000,2))
B = np.random.uniform(-3,3,(1000,2))

def distance(a,b):
return np.sum((a-b)**2)

def resample(A, B, N):
newsample = np.zeros((N,2))
for i in range(N):
# step 1
a = A[np.random.randint(0,len(A))]
#step 2
mindist = np.inf
mindistind = None
for j in range(len(B)):
d = distance(a,B[j])
if d < mindist:
mindist = d
mindistind = j
newsample[i]=B[mindistind]
return newsample

C = resample(A, B, 300)

fig, (ax1, ax2, ax3) = plt.subplots(3,1, sharex=True, sharey=True)
ax1.scatter(A[:,0], A[:,1], label='A')
ax2.scatter(B[:,0], B[:,1], label='B')
ax3.scatter(C[:,0], C[:,1], label='B from A')
[ax.legend() for ax in (ax1, ax2, ax3)]
plt.show()
$$$$
`
• Thank you for your answer ;) – xxxxx Feb 18 at 22:56