4
$\begingroup$

Given a regression setting with covariates $X_{n \times m}$ and response $Y_{n \times p}$ where $p>1$, i.e the responses are vector-valued or multivariate, is there a Nadaraya-Watson estimator for kernel regression in this setting?

This boils down to how the following can be computed with this form of $Y$ :

$$\frac{\sum_{i=1}^{n}K_h(x-x_i)y_i}{\sum_{i=1}^{n}K_h(x-x_i)}$$

But since above, $y_i$ is now multivariate as well, what happens to this multiplication operation in the numerator, in this generalization to multivariate responses?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The mathematical operation here allows also the use of a vector instead of a scalar. Think of it as a weighted sum of vectors:

$$\frac{\sum_i w_i \mathbf y_i}{\sum_k w_k} = \sum_i \left(\frac{w_i}{\sum_k w_k} \right) \mathbf y_i = \sum_i \tilde w_i \mathbf y_i$$

where the coefficients are given in terms of kernel functions

$$ w_i = K_h(\mathbf x-\mathbf x_i)\\[1em] \Rightarrow \quad\tilde w_i = \frac{K_h(\mathbf x-\mathbf x_i)}{\sum_{k=1}^{n}K_h(\mathbf x-\mathbf x_k)}$$

With this, multivariate Nadaraya-Watson kernel regression simply boils down to a one-dimensional regression in each dimension.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Awesome. i previously thought it would be a contradiction if we did multiple 1-d regressions as all dimensions may not have equally useful information. But it's good to know, that this is not a detrimental answer-and the right answer indeed. Thanks! $\endgroup$
    – hearse
    Commented Nov 26, 2014 at 3:26
  • $\begingroup$ @PraneethVepakomma: still, it was slightly misleading (--whereas I think you got it right). I made a mistake the definition of the coefficients. Actually it was correct as the normalization factor cancels out, but now its more convenient. $\endgroup$
    – davidhigh
    Commented Nov 26, 2014 at 8:47
  • $\begingroup$ how is the bandwidth h estimated here, in this case? Do you know of R-packages that implement this for multivariate Y? $\endgroup$
    – hearse
    Commented Nov 26, 2014 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.