I am trying to classify a topographic cross section (profile) using a machine learning method. The classification consists of 2 main classes (scarp, no scarp) and 3 sub-classes (cls1, cls2, and cls3) that correspond to easternward and westernward inclined scarps, or flat areas, respectively.

An example of profile: enter image description here

Some examples of data I extracted from this topography look like this: enter image description here

These data are normalized, and I have 16 variables that may distinguish the classes I need. The expression of these data vary from profile to profile, and instead of using a rule-based system, I would like to get the classes for each topographic data point probabilistically.

However, the machine learning method to use is uncertain (neural network (NN)? Regression Tree? or SVM?). For Regression Tree or SVM, I have re-worked the representation of the data to get the dataset 2 in Fig.2.

For the 1st representations of data (data set 1), I tried a non-linear classification approach using a NN system (built with TensorFlow). However, after training and testing, I get 80 to 85% accuracy for the first, and only 50% accuracy for the second... Note that I have 1700 profiles in total (multiplied by around 500 points), and I used only 70 profiles as training data.

My question is:

Should I consider using the 2nd set of data for classification with Regression Tree or SVM, or would you advise me to still consider the NN approach but improve it (through, e.g. more data, better representation...).

  • 1
    $\begingroup$ I am not quite the expert on neural networks, but why use so little of your data (70/6000) to fit the model? $\endgroup$ – IWS May 3 '17 at 11:03
  • 1
    $\begingroup$ Well, I manually attribute classes to the data, aided by if-else statements, and it takes a bunch of time to do it correctly... $\endgroup$ – JrCaspian May 3 '17 at 11:13
  • $\begingroup$ Since you have started on the NN journey (and invested time/effort) it probably will be worthwhile to explore it further by trying to refine the model before considering other models/approaches $\endgroup$ – Ironluca May 3 '17 at 12:19

First of all, you should keep labelling your data by hand --maybe a rough labelling first-- to train your neural network. As you have, 500*4 = 2000 features in the input space (4 variables per profile, and each profile has 500 points right?)! So 70 examples to train on is very little to learn something in such high-dimensional space...(and this holds for linear methods too, you will need a good regularization anyway).

Even if the neural networks might give you better final results after a bit of improvements, you will most likely gain better insights by trying to get good results with SVM let's say. And after all, if you manage to get good results with SVM and good insights in the data, then it may benefit for your NN approach.

So my main advise will be to reduce the dimensionality "by hands" before doing the classification (I say "by hands" to discard neural networks). Namely try to get features from your profiles that will help you classify them as scarp or no scarp : (here just random examples to foster ideas):

  • mean eastward length of all bouts
  • overall eastward length
  • variance of the bouts
  • ratio of number of eastward over flat
  • etc...

And all the above for each sub-classes in the profile (all the above collapses the spatial information you have, a bit like in your dataset 2, keep this in mind!).

Then you will be able to see which feature helped. Other than that you might use data-driven dimensionality reduction like PCA, or if you want to keep the spatial information of the profile you could think of other methods too (convolution with specific kernels?). But all this will make the subsequent classification faster, lighter and more interpretable. Finally it will surely give you ideas on how to improve the neural network (e.g. which architecture to use!).

  • $\begingroup$ Sorry, just realized I might have just charged ahead with this idea that you were classifying scrap vs no scrap for each profile.... But if you were to classify each data point on each profile in one of the three subclasses, then the problem is kind of solved already, dataset2 with SVM should be near perfect. $\endgroup$ – H. Rev. May 5 '17 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.