I am using UC Irvine ML Glass Identification dataset mentioned in a book "Applied Predictive Modelling".
I tried rudimentary logistic regression models using sklearn with and without the StandardScaler class and had 3% improvement in accuracy when cv =5.
I did use pipeline to not leak the data but i am not sure if i leaked it somehow.
lr = LogisticRegression(max_iter=10000, random_state=42) results = cross_val_score(estimator=lr,cv=5,scoring="accuracy", X = glass_data.iloc[:,0:-1], y=glass_data.iloc[:,-1] ) # had to increase max iterations as data is unprocessed, gradient descent is too slow because of it. np.mean(results)
above code had lower accuracy of .579
from sklearn.preprocessing import StandardScaler # since we did not divide our data set and are using cross validation, we need to use a pipeline # otherwise there will be data leakage from sklearn.pipeline import Pipeline clf = Pipeline([("scaler", StandardScaler()), ("lr", LogisticRegression( random_state=42))]) result = cross_val_score(clf, cv=5, scoring="accuracy", X=glass_data.iloc[:,0:-1], y=glass_data.iloc[:,-1]) np.mean(result)
this had accuracy of .6078
Also, i did try to increase max_iter to a very large number for case 1 but the cross_val_score average was always the same.