Comparing distribution of training and test dataset When we fit a model to a training dataset, our basic assumption is that the test dataset will also come from the same (or similar) distribution. How to check if the distribution of the training and test dataset is same (similar)? 
More specifically:
Let $(\textbf{x}, y) \sim \mathcal{D}$ where $\textbf{x} \in \mathbb{R^{5}}$ are independent variables and $y$ is the dependent variable and $\mathcal{D}$ is some unknown distribution. 
Case I Regression: $y \in \mathbb{R}$
Case II Classification: $y \in \{0, 1\}$
Question: For both these specific cases, how do we test if both training and test dataset is drawn from $\mathcal{D}$? Alternatively, how do we test if the distribution of test and training dataset is not statistically different?
Related Question: Does change in data distribution affect non-parametric models such as "Nearest Neighbor" as much as it would affect the parametric models? My intuition is that it should not. 
PS 1: I understand that we can't test if two datasets are from the same distribution, but we can test if they are from two statistically different distribution. 
PS 2: Since this seems a broad question, even pointers to some good articles should be fine. The ones that I found online dealt with the case of $x  \in \mathbb{R}$ instead of $\textbf{x} \in \mathbb{R}^{5}$ and did not differentiate regression from classification. Moreover it assumed a known distribution. 
 A: Case I and Case II aren't so different as you might think: just consider the joint distribution of $(\mathbf x, y)$, which you've helpfully already notated as $\mathcal D$. Then any two-sample testing procedure would work; I'm partial to the maximum mean discrepancy test described by

Gretton, Borgwardt, Rasch, Schölkopf, and Smola. A Kernel Two-Sample Test. JMLR 13(Mar):723−773, 2012.

It's particularly good if you (warning: shameless self-promotion) optimize the kernel with the technique proposed by the following paper:

Sutherland, Tung, Strathmann, De, Ramdas, Smola, and Gretton. Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy. ICLR 2017. (code)

There are also methods that propose to automatically adapt for possible changes between the training and the test distribution, perhaps with a few labeled examples in the test distribution. This field in general is known as transfer learning, and there's a fair amount of recent work in it; often it attempts to make the two distributions look similar to a two-sample test. A reasonable starting point is this 2016 PhD thesis (by a former labmate of mine), though there is also surely a lot of very recent deep learning work that isn't discussed there (and that I'm not very familiar with).
For your related question, I don't think it's the case that nonparametric models are in general more robust to distribution shift than parametric methods. You might be able to come up with examples where they are, but it's going to be highly dependent on the type of shift you assume; I'm sure you can also come up with examples where the nonparametric methods break faster than the parametric ones.
