By increasing the size of the training set the model memorize more data. Thus, will using leave one out increase the chance of overfitting?
ML model will start to "memorize" data with increasing complexity of your algorithm (and its parameters), not the size of training set.
Cross-validation is used to estimate your model performance on data that was not used to train. If you use LOOCV (k=n) then your k models will be (almost) identical. This gives you a high variance in your model evaluation and low bias regarding the final model trained on the entire data set.
Use 10-times 10-fold stratified CV if you unsure about a good value for k.
The main reason to not use leave-one-out would be that it is computationally heavy in comparison to k-fold cross validation. (Although it can be done more directly Proof of LOOCV formula)
The larger training set is in general not causing a stronger overfitting. In fact, it has the opposite effect. The noise will be canceled out more and is less likely to create overfitting (the model remembers the average of the training data). Also, in comparison to k-fold CV there are more ways in which the data is split up into training and test data, which should in general be better as well.