Prediction of n class variables

I have a historical data that has discrete variables. Let say I have data points with class labels

1, 2, 3, 4, and 5

For a given classification problem, I can use the training data and then get the trained model. Using the trained model, I can classify the labels. However, I am also interested in prediction of class labels.

My requirement is that for a given $N$ points, lets say 5 data points, I want to know what is the probability that class label 5 more likely or less likely occurs from 0 to 1. Can anyone give me any ideas on this? For example my output will be a prediction probability for 5 instances, being:

0, 0.1, 0.2, 0.3, 0.5.

This means that the first data point has zero probability for class label 5 to occur. The 5th data point has the highest probability to occur around 50%. Can anyone give me idea on how to go about this problem?

• It seems to me that the prediction value will come from the mechanism that you use to build your model. Am I understanding this correctly? – mandata May 4 '15 at 3:53
• Use softmax with 5 class outputs. It outputs class probabilities whose sum is 1. You may stack it to Neural network to construct more complex model. – yasin.yazici May 6 '15 at 14:28
• I want predicted probabilites rather than classification. If I have 3 class labels, I want to know which has more probability to occur from 0 to 1. Let's say we have class labels that robot can function or not, probability 0 means robot cannot function and probability 1 means robot can function. Anything in between 0 and 1 is the state of robot initializing its CPU. – prasanna May 11 '15 at 3:18

First let me rephrase the problem to make sure I understand it: you have 5 points, but you don't want to classify the points, you want classification probabilities.

If that's the case, that's what multinomial regression (Wikipedia) is designed to do. It's covered in several textbooks; I learned out of Categorical Data Analysis (Amazon link to 3e) by Agresti but it might not be the best book for self-study.

You can also use a neural network (blog post by Brian Dolhansky) that outputs class membership probabilities. Interesting coverage of the subject here (Ou and Murphey, 2007, Multi-class pattern classification using neural networks. Pattern Recognition 40, 4-18).

• You're correct. I want predicted probabilites rather than classification. If I have 3 class labels, I want to know which has more probability to occur from 0 to 1. Let's say we have class labels that robot can function or not, probability 0 means robot cannot function and probability 1 means robot can function. Anything in between 0 and 1 is the state of robot initializing its CPU. I am trying to figure out which model can fit this scenario. – prasanna May 11 '15 at 3:12

Not sure i understand what you are asking... You can determine the probability of getting class label 5 btwn 0 and 1 by simply adding the probability of getting 5 @ 0 & prob of getting 5 @ 1 (p(0) + p(1) = 0 + .1 = .1)

Solution: Since Prob of getting class 5 btwn 2 & 4 would be--> p(2) + p(3) + p(4) = .2 + .3 + .5 = .9 & prob of 5 btwn 0 & 1 = p(0) + p(1) = 0 + .1 = .1

.9 > .1 Thus Class label 5 is less likely to be btwn 0 & 1.