Trying to do CV for my polynomial regressor. However, for some polynomial degrees:as the polynomial degree increases, R^2 decreases (e.g., R^2 for degree 2 is 0.6 while for degree 3 is 0.28), why is that? 

    lin_regressor = LinearRegression()
    
    # pass the order of your polynomial here  
    poly = PolynomialFeatures(1)
    
    # convert to be used further to linear regression
    X_transform = poly.fit_transform(x_train)
    
    # fit this to Linear Regressor
    linear_regg=lin_regressor.fit(X_transform,y_train)

 



linear_regg.coef_

    from sklearn.model_selection import cross_val_score
        from sklearn.model_selection import KFold
        from sklearn.metrics import r2_score
        
    crossvalidation_poly = KFold(n_splits=3, shuffle=True) 
    
    #for train_index, test_index in crossvalidation_poly.split(X_normalized):
    
    
    for i in range(1,11):
        poly_cross_validation = PolynomialFeatures(degree=i)
        X_current = poly.fit_transform(X_normalized)
        model = lin_regressor.fit(X_current, y_for_normalized)
        scores = cross_val_score(model, X_current,y_for_normalized, scoring='r2', cv=crossvalidation_poly,
     n_jobs=1)
        
    
        print("\n\nDegree-"+str(i) +" polynomial: R^2 for every fold: " + str(np.abs(scores)))
              
              #+" training: " + str(np.abs(train_index))+" \ntesting: " + str(np.abs(test_index)))
    
        print('\033[1m'+"Degree-"+str(i)+ '\033[1m'+ " polynomial: Average R^2 for all the folds: " + str(np.mean(np.abs(scores))) + '\033[0m'+ ", STD: " + str(np.std(scores)))
    
    
    Degree-1 polynomial: R^2 for every fold: [0.41300831 0.45801624 0.17011995]
    Degree-1 polynomial: Average R^2 for all the folds: 0.34704816498535956, STD: 0.2860884371794798
    
    
    Degree-2 polynomial: R^2 for every fold: [0.75123033 0.85035531 0.40642591]
    Degree-2 polynomial: Average R^2 for all the folds: 0.6693371814650284, STD: 0.19025980734977752
    
    
    Degree-3 polynomial: R^2 for every fold: [0.30689692 0.1496736  0.38827092]
    Degree-3 polynomial: Average R^2 for all the folds: 0.28161381160006743, STD: 0.23675178460286633
    
    
    Degree-4 polynomial: R^2 for every fold: [0.7209975  0.40749117 0.84886534]
    Degree-4 polynomial: Average R^2 for all the folds: 0.6591180032208857, STD: 0.18542670407038087