2
$\begingroup$

I have gene expression data, I do dimensionality reduction and clustering with self organizing maps, but self organizing maps do not perform well with my data.
I want to map my data to feature space using different kernel functions and then give this data as input to self organizing maps to compare their performance.

My input data is of dimension $m\times n$, and upon applying a given kernel it becomes a square matrix, which I don't want to.

How can I map my data to feature space without changing dimension of my input matrix?

Improve this question
$\endgroup$
  • $\begingroup$ Why does a square matrix not work? Generally the only reasons you wouldn't want to use a kernel in this scenario is if you want to use your algorithm to return to the original feature space, e.g. ICA. $\endgroup$ – Jessica Collins Nov 15 '13 at 4:48
  • $\begingroup$ exactly, because I lose the interpretation when I don't return original feature space. the features are the genes and I want to have all genes in my final result. not combination of some genes as features. $\endgroup$ – user2806363 Nov 15 '13 at 6:16
  • 2
    $\begingroup$ Kernel are typically used as a way of implicitly mapping your data into a higher dimensional space. So even if you explicitly map your inputs to the new space, for example see the last few lines of the definition section of the polynomial kernel article on Wikipedia, the dimension of your data matrix will still change. Either the number of rows or columns will increase, depending on how your data is stored. $\endgroup$ – alto Nov 15 '13 at 12:32
  • $\begingroup$ so, suppose I have 1000 genes as features. I want to do clustering and dimensionality reduction. how can I kernelize my data without losing interpretation power ? finally I want to observe my 1000 genes in my final result to analysis them. $\endgroup$ – user2806363 Nov 15 '13 at 13:55
  • 1
    $\begingroup$ @user2806363 I think you need to do a bit of research regarding what you're trying to do. As it stands it seems like you're requirements are at odds with each other. In machine learning the use of kernels is typically in the context of the kernel trick. You can typically think of the mapping (done implicitly) as forming more complex features by combining individual features, which you stated in a previous comment you don't want to do. $\endgroup$ – alto Nov 15 '13 at 19:12
1
$\begingroup$

In general, kernels are not interpretable in the original feature space. The m by m gram/covariance matrix computed in the problem statement is the explicit mapping of the kernel function.

Each element represents the i,j distance between points i,j in the original feature space. This operation does not have an inverse to the original feature space. Each feature in kernel space represents the distance to each observation in your original space.

Finding a point in the original feature space is geometrically equivalent to triangulation in the basis function of your kernel. For an RBF kernel this is finding the point of intersection for m circles of radius given by the weight of the kernel feature, where each circle has an origin at the mth observation.

Depending on the goal of the analysis, k-means cluster centers might suffice.

Improve this answer
$\endgroup$
-1
$\begingroup$

Most of the kernels (assuming you mean kernel trick as in SVM) used are of an infinite dimension, you will have trouble storing them in our puny not-really-a-turning-machine-no-infinite-memory computers.

You will need to use an approximate version of the feature vectors, such as those obtained using Nystrome's method or Random Fourier Features.

Improve this answer
$\endgroup$
  • $\begingroup$ If you are actually storing the transformed feature vectors (in SVM), you're doing it wrong. Working w/implicit transformations into infinite dimensional space does not require approximations. $\endgroup$ – Stumpy Joe Pete Nov 16 '13 at 1:26
  • $\begingroup$ A) That is not necessarily true, see the large amount of resent research and application of approximation methods to SVM training. B) Training an SVM was not the question. If they want to use SOM, then they need the feature vectors. $\endgroup$ – Raff.Edward Nov 16 '13 at 1:41
  • $\begingroup$ Specifically, the ending part "How can I map my data to feature space without changing dimension of my input matrix?" is what I answered. The answer is, you create an approximation of the desired size. $\endgroup$ – Raff.Edward Nov 16 '13 at 1:43
  • 1
    $\begingroup$ I think the first sentence in your answer is a bit misleading. Most kernels do not map to infinite dimensions, but the most popular one does (RBF). I think it is more appropriate to say that most kernels map to a higher dimensional space, but usually not infinite dimensional. $\endgroup$ – Marc Claesen Jan 22 '14 at 7:35
  • $\begingroup$ But I did not say most kernels map to infinite dimensions. I said Most of the kernels used, of which RBF is clearly the majority holder. $\endgroup$ – Raff.Edward Jan 23 '14 at 0:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.