Trying to do CV for my polynomial regressor. However, for some polynomial degrees:as the polynomial degree increases, R^2 decreases, why is that? 



    from sklearn.model_selection import cross_val_score
        from sklearn.model_selection import KFold
        from sklearn.metrics import r2_score
        
    crossvalidation_poly = KFold(n_splits=3, shuffle=True) # Why are we using kfold=3 not 10; Because if you choose k=10, then you are increasing the points in validation set, so you are "using less data" to train the model
    
    #for train_index, test_index in crossvalidation_poly.split(X_normalized):
    
    
    for i in range(1,11):
        poly_cross_validation = PolynomialFeatures(degree=i)
        X_current = poly.fit_transform(X_normalized)
        model = lin_regressor.fit(X_current, y_for_normalized)
        scores = cross_val_score(model, X_current,y_for_normalized, scoring='r2', cv=crossvalidation_poly,
     n_jobs=1)
        
    
        print("\n\nDegree-"+str(i) +" polynomial: R^2 for every fold: " + str(np.abs(scores)))
              
              #+" training: " + str(np.abs(train_index))+" \ntesting: " + str(np.abs(test_index)))
    
        print('\033[1m'+"Degree-"+str(i)+ '\033[1m'+ " polynomial: Average R^2 for all the folds: " + str(np.mean(np.abs(scores))) + '\033[0m'+ ", STD: " + str(np.std(scores)))
    
    
    Degree-1 polynomial: R^2 for every fold: [0.41300831 0.45801624 0.17011995]
    Degree-1 polynomial: Average R^2 for all the folds: 0.34704816498535956, STD: 0.2860884371794798
    
    
    Degree-2 polynomial: R^2 for every fold: [0.75123033 0.85035531 0.40642591]
    Degree-2 polynomial: Average R^2 for all the folds: 0.6693371814650284, STD: 0.19025980734977752
    
    
    Degree-3 polynomial: R^2 for every fold: [0.30689692 0.1496736  0.38827092]
    Degree-3 polynomial: Average R^2 for all the folds: 0.28161381160006743, STD: 0.23675178460286633
    
    
    Degree-4 polynomial: R^2 for every fold: [0.7209975  0.40749117 0.84886534]
    Degree-4 polynomial: Average R^2 for all the folds: 0.6591180032208857, STD: 0.18542670407038087