Sampling from dataset according to distribution obtained from another dataset Suppose we have dataset $A$ with several categorical and numerical features:
$A_{cat_1}$, $A_{cat_2}$, $\ldots;$ $A_{num_1}$, $A_{num_2}$, $\ldots;$
Also we have another dataset $B$ with the same features, but probably with less categories (i.e. unique values) in features.
We want to sample from dataset $B$ according to joint distribution of $A$. How can one do it?
I was thinking in the following direction:


*

*We can LabelEncode categorical features. After that we can use  Kernel Density Estimation on both numerical and encoded categorical features. Is it the right way to estimate distribution of $A$ or maybe there is more correct procedure to estimate distribution in presence of categorical variables?  

*After we obtain KDE of $A$, we need to sample from $B$ according to that distribution. Could you describe (or maybe provide some code) how can one do it?


I admit this path itself could not be the best solution, so I am also open to better suggestions or any sources.
 A: So I think if I understand you correctly, what you are trying to do is to sample points from $B$ according to the distribution of $A$. In which case I would take a different approach to what you have laid out here.
First of all you talked about label encoding $A$ and then doing a KDE. I don't think this makes sense unless your categorical variables can really present something continuous ie colour $\in$ [red, orange, yellow, green] as opposed to pet $\in$ [cat, frog, horse]. The problem here is that if you label encode these and apply the KDE filter, unless you set a small width for the filter you will end up with a lot of probability distribution between theses discrete categories. You will also end up blurring your distributions for the other variables.
I think a better approach would be this:


*

*Randomly sample a datapoint $a$ from $A$

*Find the datapoint $b$ from $B$ which minimises some distance $D(a,b)$

*Repeat


Then all you need to do is define a distance function $D$. This could take into account both continuous values and the categorical values. 
For categorical variables perhaps you decide the partial distance $d_{cat_1} = 0\textit{  if  }(a_{cat_1}=b_{cat_1})\textit{  else  } \infty$ 
and for continuous variables maybe you could use $d_{num_1} = (a_{num_1}-a_{num_2})/\sigma_{num_1}$ where $\sigma_{num_1}$ is the standard deviation of $[A_{num_1}, B_{num_1}]$ and so on.
Then you could set $D = \sum_x d_x^2$
Here is a minimum toy example in python
import matplotlib.pyplot as plt
A = np.random.normal(0,1,(1000,2))
B = np.random.uniform(-3,3,(1000,2))

def distance(a,b):
    return np.sum((a-b)**2)

def resample(A, B, N):
    newsample = np.zeros((N,2))
    for i in range(N):
        # step 1
        a = A[np.random.randint(0,len(A))]
        #step 2
        mindist = np.inf
        mindistind = None
        for j in range(len(B)):
            d = distance(a,B[j])
            if d < mindist:
                mindist = d
                mindistind = j
        newsample[i]=B[mindistind]
    return newsample

C = resample(A, B, 300)

fig, (ax1, ax2, ax3) = plt.subplots(3,1, sharex=True, sharey=True)
ax1.scatter(A[:,0], A[:,1], label='A')
ax2.scatter(B[:,0], B[:,1], label='B')
ax3.scatter(C[:,0], C[:,1], label='B from A')
[ax.legend() for ax in (ax1, ax2, ax3)]
plt.show()
```

