I have a bit to learn about machine learning, so please pardon me if I am asking the wrong type of question. I have read some about neural networks and SVMs, so I'm not completely in the dark.
I am wondering how to tell how may dimensions a problem has: is it the number of possible outcomes or the total number of inputs? Does it depend on the type of machine learning algorithm or only on the particular problem at hand? Or am I missing something entirely?
*Most literature I have read refers to 'high-dimensionality', but I am wondering if an exact number of dimensions can be calculated. I am then hoping to use this when trying to reduce the number of dimensions (when I get that far) to judge the overall effectiveness of a strategy. But first I must better understand the dimensions of a machine learning problem.
**If necessary, please use neural networks or SVMs as a reference point, but I am also interested in hearing about genetic algorithms and anything else you might like to mention.