I'm interested in doing distribution regression with Kernel Mean Embedding with a variety of kernel methods (SVM regression, Ridge Regression, Gaussian Process). Typically in distribution regression, you have something like that maps $x \to y$. $x$ is typically a distribution (i.e. 10 people have spend \$5, 13 people spend \$6... and convert that to a probability distribution). My question is lets say you have a distribution that you think is correlated with y. However, you also have other variables like location that is also correlated with y that is not a distribution. I'd like be able to input both the distribution and real value variables into my algorithm and use the combination of the two to predict y. Obviously there are work arounds like building two models and ensembling, but I'm more interested in doing the optimization/modeling end to end.
One way to combine the mean embedding and other covariates is through additive kernel. E.g., eqn 15 of http://sethrf.com/files/ecological.pdf
So for anyone looking for a solution if you assume non-distribution features are kroneker delta distributed with constant value, you can take expectations of the distributions while keeping KD constant values the same.