I am a beginner at machine learning and am trying to implement a basic logistic regression using the formula for the cost function we learned in class:
https://miro.medium.com/max/2836/1*_52kKSp8zWgVTNtnE2eYrg.png (cost function) https://miro.medium.com/max/2964/1*o4Dy1w4n2kDOLA8UEwGC9g.png (hypothesis function)
I implemented the following code, and it looks like depending on my initial guess, fminunc usually finds a local minimum near to the guess, but also sometimes stops for other reasons (such as "undefined at initial point" or "cannot decrease the objective function along the search direction") indicating there are many local minima in addition to potentially other issues. I also have similar issues with fminsearch, and updated the code below with fminsearch. If anyone with more experience could provide some insight, it would be greatly appreciated.
Here is my code:
%logistic regression: predicting benign vs. malignant tumors with data on %x0=1, size, patient age, and bloodflow rate clear global x y; x=[1,7,50,42;1,77,77,2;1,87,75,1;1,87,70,21;1,60,91,10;1,80,90,3;1,30,20,23;1,10,25,20;1,15,20,33;1,10,20,23;1,50,60,1;1,10,20,15;1,40,70,5;1,10,10,24;1,80,80,1]; y=[0;1;1;1;1;1;0;0;0;0;1;0;1;0;1]; fminsearch(@cost,[-1,.1,.1,.1]) function f=cost(a) global x y; f=0; e=2.71828; for i=1:length(y) f=f+(y(i)*log(1./(1+e^(-a*x(i,:)')))); f=f+((1-y(i))*log(1-(1./(1+e^(-a*x(i,:)'))))); end f=-f./length(y); end ```