Although you might think that a model with only 6 features is low in complexity, it's too complex for the data at hand. You do not have enough data to fit a logistic regression model reliably with 6 unpenalized features.
The high variance/low bias behavior you observe on multiple random train/test splits is what is expected with an overfitted model: too many parameter values estimated for the number of cases available. The usual rule of thumb for avoiding overfitting in unpenalized logistic regression is to have 10-20 cases of the least prevalent class per feature considered. With only 14 cases total you have no more than 7 in the least prevalent class, so you would be hard pressed to evaluate even a single unpenalized feature reliably.
If you were nevertheless to go on to investigate transformations of predictors (e.g., splines or other polynomial fits) to try to improve performance, you would be compounding your problems. In applying that rule of thumb for avoiding overfitting, you need to count each additional estimated parameter as an additional feature. So without additional precautions you would be heading even further down the overfitting path. As John von Neumann is alleged to have said: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk."
One possible way around this problem would be to use a penalized approach like ridge regression or LASSO. These approaches deliberately trade off increased bias for reduced variance by lowering the magnitudes of regression coefficients or (in LASSO) removing some features completely. That could, in principle, be combined with evaluation of polynomial or other transformations of features. With only 14 cases total, however, it would be difficult to convince a skeptical reader that your conclusions were correct.
With respect to hyperparameter choices, remember that standard unpenalized logistic regression has no hyperparameters to choose. It provides a model of class-membership probabilities. You only produce a decision boundary when you impose a probability cutoff for predicting class membership, based on some decision about relative costs and benefits of different misclassifications. That choice of cutoff might be considered a hyperparameter, but that comes after the logistic regression per se.
You do have hyperparameter choices to make if you choose the complexity of feature transformations by cross-validation or if you use penalization. In either case, however, the predictor function provided by the logistic regression model will still be linear in the coefficients of the features, however the features were transformed and however the coefficients were penalized. So the nature of the ultimate decision boundary remains the same as for standard logistic regression even if its location, the dimensions of its feature space, or its direction in feature space are different. Is that still a "first degree decision boundary" if some of the features have been non-linearly transformed from their originally observed values? I wouldn't worry too much about that terminology.
See section 4.2 of Harrell's Regression Modeling Strategies or of his associated class notes for guidance on cases per feature in different types of regressions. Chapters 6 and 7 of An Introduction to Statistical Learning provide one introduction to penalized methods. See Chapter 7 of The Elements of Statistical Learning for extensive discussion of what types of error you can estimate with these approaches even if you had more data. An important and difficult question is how well your "test error" would generalize to a new sample from the same underlying population.
Where to go from here
You want to "provide a proof-of-concept of application of Machine Learning in my research field." So you have to set up tests that will fairly represent how machine learning might be applied.
Note that with 14 cases and 6 features it is possible to include all predictors in an unpenalized linear model. The theoretical impossibility of an unpenalized linear model arises when the number of features exceeds the number of cases. So your model including all features is not violating a strict "high dimensionality" cutoff, although the unpenalized model will probably be overfit.
Also, there are 3 potential problems in the way you are performing cross validation.
First, leave-one-out cross validation is not always a good choice, as discussed in detail on this page.
Second you seem to be using classification error as the measure of model quality. If so, that is probably not wise as misclassification error depends on your particular choice of a cutoff for the probabilities returned by your logistic regression model. A proper scoring rule like the Brier score is probably a better choice. It's best to first optimize the probability predictions, then apply whatever cutoff makes sense for classification (if classification is necessary). Although the Brier score is the mean-square error between predicted probabilities from the logistic model and the actual (0,1) values, note that no logistic regression is a "least squares fit," as a logistic model has no closed-form solution and the fit is based on maximum likelihood.
Third, it's not clear whether you standardized your continuous features before the ridge regression analysis. As the ridge penalty is applied to the sum of squares of all coefficients, without standardization you get different results if a predictor based on length, say, is measured in miles versus millimeters. (Some software does this automatically for you, then transforms coefficients back to the original scales.)
When studies in your field involve such a low ratio of cases to predictors, then there is not much hope of doing substantially better than what you already have done. As you might expect, the more complex models tend to require more penalization and it's not really clear that any of them will outperform the standard unpenalized (and likely overfit) model even if you use a proper scoring rule like the Brier score.
You could, however, consider using your knowledge of the subject matter to design simulations of data sets (with appropriate choices of coefficients and error terms) to evaluate how machine learning techniques might be of value in larger-scale studies.
