0
$\begingroup$

I am using raw data set with 4 feature variables (Total Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, and Cigraeette count) to do a Binominal Classification (find stroke likelihood) using Logistic Regression Algorithm.

I made sure that the class counts are balanced. i.e., an equal number of occurrences per class.

using Python + sklearn, the problem is that the classification performance gets very negatively-impacted when I try to normalize the dataset using

X=preprocessing.StandardScaler().fit(X).transform(X)

or

X=preprocessing.MinMaxScaler().fit_transform(X)

So before normalizing the dataset:

            precision    recall  f1-score   support

  1           0.70      0.72      0.71        29
  2           0.73      0.71      0.72        31
avg / total   0.72      0.72      0.72        60

while after normalizing the dataset (the precision of class:1 decreased significantly)

           precision recall f1-score support

  1         0.55      0.97      0.70        29
  2         0.89      0.26      0.40        31
avg / total 0.72      0.60      0.55        60

Another observation that I failed to find an explanation to is the probability of each predicted class.

Before the normalization:

  [ 0.17029846  0.82970154]
  [ 0.47796534  0.52203466]
  [ 0.45997593  0.54002407]
  [ 0.54532438  0.45467562]
  [ 0.45999462  0.54000538]

After the normalization ((for the same test set entries))

  [ 0.50033247  0.49966753]
  [ 0.50042371  0.49957629]
  [ 0.50845194  0.49154806]
  [ 0.50180353  0.49819647]
  [ 0.51570427  0.48429573]

Dataset description is shown below:

       TOTCHOL    SYSBP    DIABP  CIGPDAY   STROKE
count  200.000  200.000  200.000  200.000  200.000
mean   231.040  144.560   81.400    4.480    1.500
std     42.465   23.754   11.931    9.359    0.501
min    112.000  100.000   51.500    0.000    1.000
25%    204.750  126.750   73.750    0.000    1.000
50%    225.500  141.000   80.000    0.000    1.500
75%    256.250  161.000   90.000    4.000    2.000
max    378.000  225.000  113.000   60.000    2.000

SKEW is

 TOTCHOL    0.369
 SYSBP      0.610
 DIABP      0.273
 CIGPDAY    2.618
 STROKE     0.000

Is there a logical explanation for the decreased precision?

Is there a logical explanation for the very-close-to-0.5 probabilities?

Given that I can not use this range of probabilities, can I neglect the normalization step?

Improve this question
$\endgroup$
13
  • 2
    $\begingroup$ Why would you want to scale inputs with the Logistic Regression model? It should work well on raw data, and moreover, you can interpret the results more clearly than when you transform input data. $\endgroup$
    – alexeymosco
    Commented Sep 9, 2019 at 15:12
  • $\begingroup$ Thanks for the reply @AlexeyBurnakov , I read that scaling is required when features have different ranges (please refer to the above post to compare the ranges), hence I thought I should use it. $\endgroup$
    – GYSHIDO
    Commented Sep 9, 2019 at 15:26
  • 1
    $\begingroup$ That is advised to do for a neural network model, because weights will saturate much-much faster, compared to raw data of different ranges. However, you need to understand by which method your model is being fit, MLE/gradient descent. MLE is solved without scaling. $\endgroup$
    – alexeymosco
    Commented Sep 9, 2019 at 15:30
  • $\begingroup$ Hi @AlexeyBurnakov , I am using 'sag' (Stochastic Average Gradient) >>> LR = LogisticRegression (C=0.001, solver = "sag",max_iter=1000).fit (X_train, y_train) $\endgroup$
    – GYSHIDO
    Commented Sep 9, 2019 at 15:47
  • 1
    $\begingroup$ Point is, deal with regularization, it causes these effects. $\endgroup$
    – alexeymosco
    Commented Sep 9, 2019 at 18:57

1 Answer 1

Reset to default
0
$\begingroup$

So I have found the answer to my own question, went back to the Python script and in the command that fits the model i.e. 

 LR = LogisticRegression (C=0.1, solver = "sag",max_iter=1000).fit (X_train, y_train)

The parameter C was set to 0.001 which is a very small value (meaning lambda is very high as C=1/lambda) (C is the regularization strength and smaller values indicate stronger regularization). more on that matter can be found here and here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.