Why is this learning curve changing when the training sizes don't start with 1 sample?

Question

I started following the tutorial to create a linear regression model with scikit from here inside jupyter notebook.
Then I decided to do a k-cross validation and plot the learning curves.
The code I wrote is this one

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import learning_curve
%matplotlib inline
...
# Load dataset, prepare the dataset, show dataset infos...
...
# Prepare training and testing set with 80:20 ratio
test_size = 0.2
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_size, random_state=42)
print("DATA SIZES from total of", X.size)
print("Training size: ", 100-test_size*100, "% of", X.size, "=", X_train.size)
print("Test size: ", test_size*100, "% of", X.size, "=", y_test.size)

# Prepare various training sizes for cross validation
train_sizes_1 = np.array([1, 19047, 38093, 57139, 76185])
train_sizes_2 = np.array([0.00001, 0.12, 0.325, 0.55, 0.775, 1.])
train_sizes_3 = np.array([0.001, 0.12, 0.325, 0.55, 0.775, 1.])

# Do k-cross validation and save learning/validation errors
regressor = LinearRegression()
val_train_sizes, val_train_scores, val_scores = learning_curve(
    regressor, X_train, y_train, train_sizes=train_sizes_1, cv = 6, scoring='neg_root_mean_squared_error')

val_train_scores_mean = -val_train_scores.mean(axis=1)
val_scores_mean = -val_scores.mean(axis=1)
print('Mean training scores\n\n', pd.Series(val_train_scores_mean, index=val_train_sizes))
print('\n', '-' * 20)
print('\nMean validation scores\n\n', pd.Series(val_scores_mean, index=val_train_sizes))

learning_curves_figure = plt.figure(figsize=(10,6))
plt.title("Regression Model Learning Curve")
plt.xlabel("Training set size")
plt.ylabel("RMSE")
plt.plot(val_train_sizes, val_train_scores_mean, "o-", color="g", label='Training error')
plt.plot(val_train_sizes, val_scores_mean, "o-", color="r", label='Validation error')
plt.legend()
...
# Fit the model, predict test and show model infos...
...

train_sizes_1(with manually selected sizes) and train_sizes_2(with percentages) both start with 1 sample(I took the idea from here) to use for the first cross-validation while test_sizes_3(with percentages) starts with multiple samples.
What happens with train_sizes_1 and train_sizes_2 is that the learning curves are pretty much like this

but with train_sizes_3 I get this

My questions are:

1. Are the curves actually telling something that I'm not understanding or there's a programming error misleading me ?
2. If the curves are not wrong, what is actually happening with the changing of the first training size ?

Answer 1

There is nothing wrong with the code. As shown in your article, with only one sample for the training, you'll have no difficulties to find a slope that intercept your sample. The training error is then O but the validation error is high (high bias & high variance).

Answer 2

Looks like I made a sight mistake.
When looking at the learning curves for train_sizes_1 and train_sizes_2 I thought the graphs were pretty much the same because the training and validation points were too close to each other and I uploaded the learning curves of train_sizes_1.
The problem is that:

1. train_sizes_1 and train_sizes_2 don't have the same number of sizes
2. train_sizes_1 last size isn't the whole training set

So I made a k-cross validation with train_sizes_2 and the right learning curves are

Now you can slightly see that the training error gets higher than the valiation error and than decreases until it overlaps with the validation error(like what happens with train_sizes_3).
As a matter of fact, the mean training and validation errors for train_sizes_2 are

Mean training scores   
 1       -0.000000
9523     4.196595
25792    4.188449
43648    4.178641
61504    4.174143
79360    4.174099

--------------------

Mean validation scores
 1        12.748529
9523      4.174330
25792     4.174154
43648     4.174196
61504     4.174180
79360     4.174185

and for train_sizes_3 are

Mean training scores
 79       4.163410
9523     4.196595
25792    4.188449
43648    4.178641
61504    4.174143
79360    4.174099

--------------------

Mean validation scores
 79       4.222444
9523     4.174330
25792    4.174154
43648    4.174196
61504    4.174180
79360    4.174185

When using 1 sample as a starting size the errors were too high and the next ones were too low so the matplotlib graph couldn't show the visual difference that well.