1
$\begingroup$

I designed a neural network for binary classification in MATLAB R2015a.

  • What are differences between two classes? How system detects a sample is from class 1 or 2? For example some if,then functions or some ranges. Something which can help user to have clearer view of the results. For example neural network found that a sample is from class 1. I should say in report that what are differences between in this sample and other samples (class 2) for this results (based on neural network inputs). I know this is a black box but I need more results. I'm checking the trained system using out-of-sample data.

  • I want know the effects of inputs on output. Which input (feature) is more impotent (has higher weight) on output of trained neural network.

PS. Suppose that i have 5 inputs (features) that i checked we have higher accuracies with these input's combination. So now i want find class 1 and class 2 characteristics. My neural network has two hidden layers. First one has 5 neurons and second one has 3 neurons. My hidden layers transfer function is 'tansig' and output transfer function is 'softmax' for reporting probabilities for outputs. Now what should i do ?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

My first suggestion would be to use Logistic Regression (LR), at least to gain some understanding regarding input feature relationships to output. LR provides a lot more information than neural nets in this regard, since neural nets are much more black boxes.

While neural networks tend to be black boxes, there are few things things that you can do, depending upon the type of neural network and the problem that you are solving. As an example, consider the image classification. Often the weight vectors for each neuron are examined to look for detection patterns. The results are generally edge detectors, etc.

https://a248.e.akamai.net/f/1097/1823/7m/deliveryimages.acm.org/10.1145/2010000/2001295/figs/f3.jpg

Similar information is often shown by researchers in a dashboard that is presented at classification time.

http://www.codeproject.com/KB/library/NeuralNetRecognition/Screenshot-GraphicalView.gif

In this image, there is also a visual indication of the output by class, indicating likelihood by class (8 being most likely in this image).

You might consider showing examples of inputs that your neural network gets consistently right and wrong when the predicted class is indicated. This gives the user an idea of the types of errors that are likely for the prediction that is made.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you John. Can you describe more about your second solution? Suppose that i have 5 inputs (features) that i checked we have higher accuracies with these input's combination. So now i want find class 1 and class 2 characteristics. My neural network has two hidden layers. First one has 5 neurons and second one has 3 neurons. My hidden layers transfer function is 'tansig' and output transfer function is 'softmax' for reporting probabilities for outputs. Now what should i do ? $\endgroup$
    – user2991243
    Commented Jun 17, 2015 at 15:02
  • 1
    $\begingroup$ The two things that I can think of to represent what the neural net is doing in your case is: 1. Examine the body of predictions that the neural net is getting wrong and present some examples to the user of these tricky cases. 2. Provide some visual representation of your softmax output. Since this is a binary classifier, you might just represent the output as a probability/certainty level. This will allow the user to better understand whether your classifier is relatively confident, or not. Beyond this, describing what the neural network is doing, internally, will likely be very difficult. $\endgroup$
    – John Yetter
    Commented Jun 17, 2015 at 16:03
  • $\begingroup$ So you think this is a good solution to compare two output classes based one inputs. For example distribution of first feature for class 1 and class 2 and compare these distributions. distribution of second feature for class 1 and class 2 and compare these distributions (or ranges etc.). Based on these results I can find characteristics and differences of two classes? What other things i can do for this propose? $\endgroup$
    – user2991243
    Commented Jun 17, 2015 at 19:40
  • $\begingroup$ I don't think that you will find characteristics of your class, but if you look at the group of items for which your system gets predictions wrong, you may be able to improve your feature engineering, and you may be able to give your users examples of the types of categories that your system finds difficult to differentiate. For example, in recognizing digits, 2's with a small bottom-most line might be confused for 7's. $\endgroup$
    – John Yetter
    Commented Jun 18, 2015 at 18:33
0
$\begingroup$

Regarding the first point, you can either use linear methods that are more explanatory. For example PCA, to visualize and see what feature leads to the increase in variation. Or alternatively a linear model which each coefficient corresponds to each feature. The value of each coefficient is an indicator of how much they are influential.

Regarding the second point, you can use pruning strategy to pin point which layers, inputs are most influential. For example you can use the the prune function. So at every step you remove a node you can calculate the MSE or the loss and see which node in each layer causes the most lost. Alternatively, you can use a linear model like Lasso, as a linear approximation to the Neural Network fit.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.