2
$\begingroup$

I have a distance matrix with some noise (e.g. obtained by asking people how similar two objects from a set of objects are). I am interesting in finding the (best guess for the) dimensionality of the feature space for the objects (i.e. how many features of the objects people are presumably taking into account when rating similarity).

I know of algorithms that reduce the dimension to a given (e.g. multidimensional scaling) but I don't know how to guess the number of dimensions given that there is some noise.

$\endgroup$
  • 1
    $\begingroup$ Is the distance matrix your only information about the objects? If so I think it will be impossible to recover how the distances came to be or even make any sensible inference about it. $\endgroup$ – einar Nov 28 '17 at 18:06
  • $\begingroup$ I find your question in the last paragraph strange. Doesn't MDS have no ways to suggest the number of dimensions? Scree plot (like in PCA), Rsq, stress plots... $\endgroup$ – ttnphns Nov 28 '17 at 19:33
  • $\begingroup$ If your question is about the maximal number of dimensions in eucl. space which accomodates your data cloud assuming your distances are noised euclidean ones, this is equal to the number of positive eigenvalues of the double centration matrix stats.stackexchange.com/a/12503/3277. $\endgroup$ – ttnphns Nov 28 '17 at 19:39
  • $\begingroup$ I asked a similar question here: stats.stackexchange.com/questions/257389/… . I found the paper by levina and bickel to be helpful. $\endgroup$ – David Kozak Nov 28 '17 at 20:52
1
$\begingroup$

To answer this question in general is very hard. The "recommended" dimensionality highly depends on the data and the noise.

In general:

  • If you choose your dimensionality too high -> over fitting will occur
  • If you choose your dimensionality too low -> you may lose important dimensions

I would recommend tinkering a little bit with PCA (Principal component analysis) to get a "feeling" of the data. Also plotting the data raw as well as of dimensionality reduction approaches you often see (as a human) patterns emerge which will be in certain dimensions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.