I am a bit concerned, as there are so many questions asked and so few answers given. I take it, machine learning has become quite common to use, but only little is really understood about their nature. That's problematic, but on the other hand - yes, I am one of those people. Not enough knowledge, but bound to use them and now puzzling over my results which seem quite chaotic to me.

This is the problem: I use Support Vector Algorithms for regression of remote sensing data (satellite data in pixels). I want to compare the capability of one sensor to predict paramaterts like e.g. chlorophyll content of plants. I have the sensor data and the parameters, so I can calibrate and validate the model.

Now one sensor has 13 bands (13 features), the other one has 242 bands. They cover the same spectrum, only one has a higher spectral resolution. In theorey I'd suppose: the more features available, the better my result. Turns out, this is not the case. The 13 features deliver way more accurate parameters! The more features I add to the algorithm, the worse it gets. Another disturbing fact is that a feature selection showed me, there was a constant improvement of the results when I used 100 features and more. But I never reach the goodnes of fit for the 13 band sensor.

Is that a problem of the SVR or am I mistaking something? Thanks for your help all, very much appreciated!

  • $\begingroup$ Some parameters are here to control the sparsity of each model. Did you try a model with a large number of features and a large "C" (the cost parameter)? $\endgroup$
    – RUser4512
    Commented Mar 4, 2016 at 12:49
  • $\begingroup$ The algorithm is implemented in an IDL toolbox with JavaBridge. In the GUI I can specify the C-parameter and g (since it's a gaussian kernel) in a range and declare the step - it will take the best within the range (according to the performance) and visualize them so I can be sure it's not a local minimum. The number of features I tried was 242 (all of them) or fewer. But it seems, the SVM can't "handle the choice" which seems a bit strange to me. I always get better result for the sensor which only has 13 features available to choose from. $\endgroup$ Commented Mar 4, 2016 at 12:55
  • $\begingroup$ Echoing RUser4512 it is certainly $ \textit{possible } $ that with the C you are using you are overfitting the data. A small value of C could be the answer. $\endgroup$
    – meh
    Commented Mar 4, 2016 at 20:43
  • $\begingroup$ Thanks for pointing out the risk of overfitting! I reduced C, but all that happens, is an overall decrease in model certainty. The 13 feautures are still performing better, only both data sets now generate wrong results. :-/ $\endgroup$ Commented Mar 7, 2016 at 12:49


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.