0
$\begingroup$

Given a simple data set to train with neural networks where i.e.: wine quality is the categorical output and measurements of acidity, sugar, etc. are the numerical inputs.

The output can be written as a number i.e.: 1-10 and treat the problem as a regression model, or encode the output in 10 different columns with 1 or 0 for each corresponding quality level - and therefore treat the problem as a classification model.

Sample data:

enter image description here

What is the right answer? How to determine how to approach problems like this?

$\endgroup$
  • $\begingroup$ My guess is classification but I need some scientific rational such as for a regression you need a unique value for each pair ${x, y = f(x)}$. $\endgroup$ – Edv Beq Jan 11 at 19:42
0
$\begingroup$

To your first point, you should not treat this problem as a regression one. The numbers 1-10 are being used to classify the wines into categories, and running a regression analysis will simply give you spurious output that has no meaning.

The best way to treat this is as a classification problem, since the purpose of this exercise is to classify the wines into separate categories.

Since you are training a neural network, the first task is to normalize the data. This is necessary for the neural network to have a common scale to interpret the output, otherwise the results will be spurious.

I am using R for this example, but this could be accomplished with max-min normalization as follows:

#Max-Min Normalization
normalize <- function(x) {
  return ((x - min(x)) / (max(x) - min(x)))
}

maxmindf <- as.data.frame(lapply(mydata, normalize))
attach(maxmindf)

For this example, I split the data into training and test, and used a (5,5) configuration to train the model with a 0.046 error:

#Training and Test Data
trainset <- maxmindf[1:100, ]
testset <- maxmindf[101:178, ]

#Neural Network
library(neuralnet)
nn <- neuralnet(wine ~ alcohol + malicacid + ash + alcalinity + magnesium + phenols + flavanoids + nonflavanoid + proanthocyanins + color + hue + od + proline, data=trainset, hidden=c(5,5), linear.output=FALSE, threshold=0.01, err.fct = "sse", act.fct = "logistic")
nn$result.matrix
plot(nn)

Here are some sample results and a visual interpretation:

> nn$result.matrix
                                           1
error                         0.046648449255
reached.threshold             0.009349419960
steps                       231.000000000000
Intercept.to.1layhid1         0.483625433010
alcohol.to.1layhid1          -1.468753801323
malicacid.to.1layhid1        -0.929985197517
ash.to.1layhid1              -0.044339445361
...
1layhid.4.to.2layhid5        -1.903071654581
1layhid.5.to.2layhid5        -0.018605013954
Intercept.to.wine            -0.744974377985
2layhid.1.to.wine             7.280155712308
2layhid.2.to.wine             2.762637258209
2layhid.3.to.wine            -5.718285548975
2layhid.4.to.wine             0.462473919655
2layhid.5.to.wine            -1.786270757017

Neural Network

enter image description here

Now that the network has been built, the next step is to test the resulting output against the test set, and use what is known as a confusion matrix to determine how accurately the model classifies the wines.

> nn.results <- compute(nn, temp_test)
> results <- data.frame(actual = testset$wine, prediction = nn.results$net.result)
> results
    actual      prediction
101    0.5 0.2330174892958
102    1.0 0.9950191304006
103    1.0 0.9994186571103
104    1.0 0.9987668317395
105    0.5 0.5910452601308
.....
174    1.0 0.9978374795947
175    0.0 0.0020009597702
176    0.0 0.0036695908245
177    0.0 0.0100022572455
178    0.5 0.3734674904247
> roundedresults<-sapply(results,round,digits=0)
> roundedresultsdf=data.frame(roundedresults)
> attach(roundedresultsdf)
> table(actual,prediction)
      prediction
actual  0  1
     0 39 16
     1  0 23

From the above, we can see that of the 78 test observations, 62 of them are indicated to have been classified correctly - giving us an accuracy rate of nearly 80%.

As has been mentioned, this is assuming that there is no order between the observations in the dependent variable. If there are, it may be possible to use a regression-based neural network, but the danger is that your model would not have enough variation in the dependent variable (since there are only 10 values), and classification may be a better solution altogether for this reason.

$\endgroup$
  • $\begingroup$ Could you please tell me what the structure of you network was and what activation functions did you use? I would like to compare with one of my tries with the exact parameters. Thank you $\endgroup$ – Edv Beq Jan 17 at 23:41
  • $\begingroup$ Sure. As you'll see from the above, we have 13 input variables with a (5, 5) hidden configuration. The activation function is "logistic", and the error function is "sse". $\endgroup$ – Michael Grogan Jan 20 at 7:49
  • $\begingroup$ Thank you for your reply. Just to clarify - Both your hidden layers and including the output layer were activated with the logistic sigmoid function? $\endgroup$ – Edv Beq Jan 20 at 14:46
0
$\begingroup$

You shouldn't convert categorical variables into numeric since it induces some kind of order between them, which you don't actually know, i.e. proximity of 1 and 2, and 1 and 10 are different in regression but the same in one-hot encoding.

EDIT: If your dataset description (or your know-how) doesn't say anything about a predefined mathematical order between the classes, I mean those number could have been 10 to 1, instead of 1 to 10, or any random assignment of distinct numbers; then, you shouldn't do a regression, because solving for regression makes an implicit assumption about the order between the class labels, which probably doesn't exist at all.

EDIT 2: since wine quality assumes an order, which I didn't realize at first, yes, regression seems very reasonable since otherwise you'll lose the order.

$\endgroup$
  • $\begingroup$ The dataset has numbers 1-10 for the output. My questions were 1) Do I keep numbers or do I convert to binary? and 2) Depending on how we answer the first question - does it make the problem a regression or classification? I don't understand your answer. $\endgroup$ – Edv Beq Jan 11 at 20:02
  • $\begingroup$ I added some more explanation, regarding your comment. Hope it helps. $\endgroup$ – gunes Jan 11 at 20:21
  • $\begingroup$ I agree with gunes in general, but for the specific example of wine quality given here, assuming the values 1 - 10 represent some score and therefore some order seems reasonable to me. $\endgroup$ – nope Jan 11 at 20:32
  • $\begingroup$ I haven't realized that it was wine quality, I've added hopefully the last edit :) $\endgroup$ – gunes Jan 11 at 20:46

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.