2
votes
$\begingroup$

Probability distribution of two classes is given by $N(5,1)$ and $N(6,1)$ where $N(\mu,\sigma^2)$:

$$f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} $$

  • How to classify them, and see error rate?

I am doing this in MATLAB

Taking 500 samples of each distribution and tagging them depending where they came from.

sample1 =    
    0.8864   -1.0000
    0.1560   -1.0000
    0.8502   -1.0000
   -0.4059   -1.0000
    0.9298   -1.0000
sample2 =
   -0.0671    1.0000
    0.7057    1.0000
    0.3310    1.0000
   -0.7314    1.0000
   -0.4524    1.0000
data =
    0.8864   -1.0000
    0.1560   -1.0000
    0.8502   -1.0000
   -0.4059   -1.0000
    0.9298   -1.0000
   -0.0671    1.0000
    0.7057    1.0000
    0.3310    1.0000
   -0.7314    1.0000
   -0.4524    1.0000

Now I would like to classify them using Bayes, I am using normpdf, to make this easier I am taking prior probabilities equal so they are not important in creating the rule, but I do not know how to code this in MATLAB, any idea?

n=500;
sample1=[randn(n,1) -1*ones(n,1)];
sample2=[randn(n,1) ones(n,1)];
data=[sample1; sample2];
mu1    =5;
sigma1 =1;
mu2    =6;
sigma2 =1;
x1=linspace(mu1-1*sigma1,mu1+1*sigma1,500);
p1=normpdf(x1,mu1,sigma1);  
x2=linspace(mu2-1*sigma2,mu2+1*sigma2,500);
p2=normpdf(x2,mu2,sigma2);
plot(x1,p1,x2,p2)

Also Is it correct to label with -1 and 1? If I compute mean and variance of sample1, and sample2? mean_S1=mean(sample1); mean_S2=mean(sample2); var_S1 = var(sample1); var_S2 = var(sample2); - What is next step? - for error rate Im planning to do a comparison between the original class vector (-1,1) and the result of classifier like:

errorRate = mean(OriginalClasses ~= ResultOfClassifier);

**UPDATED**

clear;
clc;
n = 500;
mu1    =5;
sigma1 =1;
mu2    =6;
sigma2 =1;
mu = [mu1,mu2];sigma = [sigma1,sigma2];  %group them
%suppose you get your test data from somewhere. 
%for kicks, I put random data in:
%xtest = randn(2*n,1);     %OP example code has the labels in the data var; ack
sample1=[randn(n,1) -1*ones(n,1)];
sample2=[randn(n,1) ones(n,1)];
data=[sample1; sample2];
deviance = bsxfun(@minus,data,mu);  %tbc
deviance = bsxfun(@rdivide,deviance,sigma); %tbc
deviance = deviance .^ 2; %tbc
deviance = bsxfun(@plus,deviance,2*log(abs(sigma))); %tbc
deviance = deviance + log(2*pi);  %not necessary here, actually;
[dummy,mini] = min(deviance,[],2);  %find which class;
 ResultOfClassifier = 2 * mini - 3;  %is now a -1/1
 errorRate = mean(data(:,2) ~= ResultOfClassifier)

errorRate =

    0.2680
$\endgroup$
2
votes
$\begingroup$

Here's how I have done this in matlab:

mu = [mu1,mu2];sigma = [sigma1,sigma2];  %group them
%suppose you get your test data from somewhere. 
%for kicks, I put random data in:
xtest = randn(2*n,1);     %OP example code has the labels in the data var; ack
deviance = bsxfun(@minus,xtest,mu);  %tbc
deviance = bsxfun(@rdivide,deviance,sigma); %tbc
deviance = deviance .^ 2; %tbc
deviance = bsxfun(@plus,deviance,2*log(abs(sigma))); %tbc
deviance = deviance + log(2*pi);  %not necessary here, actually;
[dummy,mini] = min(deviance,[],2);  %find which class;
ResultOfClassifier = 2 * mini - 3;  %is now a -1/1
$\endgroup$
  • $\begingroup$ Could you please explain why you use these lines of code deviance = bsxfun(@minus,xtest,mu); %tbc deviance = bsxfun(@rdivide,deviance,sigma); %tbc deviance = deviance .^ 2; %tbc $\endgroup$ – edgarmtze Mar 21 '11 at 18:33
  • $\begingroup$ @darkcminor: the xtest that I have, and the sample data you provide are drawn from $\mathcal{N}(0,1)$. The classifier is always going to prefer the $\mathcal{N}(5,1)$ class to the $\mathcal{N}(6,1)$ class in this case! Instead, why not try comparing to two classes, one with mean $-1$ and the other with mean $1$. $\endgroup$ – shabbychef Mar 21 '11 at 18:34
  • $\begingroup$ Guess not getting the complete idea, I understand N(5,1) will be preffered to N(6,1), How could you incorporate Prior probabilities, as they are lets say ( p(x∣c1))/(p(x∣c2)) = .90/.10 Do you just multiply?, I guess I am missing a logarithm here, Guess log(.90/.10) $\endgroup$ – edgarmtze Mar 21 '11 at 18:40
  • $\begingroup$ if $\log(P(x = c_1) / P(x = c_2))) > 0$, then call it class $1$, otherwise call it class $2$. $\endgroup$ – shabbychef Mar 21 '11 at 18:47
  • $\begingroup$ kindly please why use log ?? $\endgroup$ – user57368 Oct 10 '14 at 12:38
1
vote
$\begingroup$

It is possible to use normpdf function for both populations and then compare results:

decision_vector = arrayfun(@gt, normpdf(X1, mu, sigma), normpdf(X2, mu, sigma));

It creates logical array where 1 means the data are from X1, 0 otherwise.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.