You will get an estimate of Pearson and Spearman correlation with as few as two distinct values, meaning that your data can be shown as two distinct points on a scatter plot. The estimate of correlation will be $1$, $-1$ or $0$ and it is quite possible that such a correlation is a reasonable answer.
(In this circumstance of two distinct data points, Pearson and Spearman correlation give identical answers; that is not true in general.)
In principle, we can only say how reasonable a correlation is if we have (a) a good idea of the pattern of the rest of the data (b) a good understanding of the behaviour of the system in question. In practice, naturally, (a) or (b) or both may be lacking.
So, there is no absolute minimum apart from that limit of 2. But any instinct to distrust results from a small sample is quite right. Much of, if not most, twentieth-century statistics pivots on distrust of small samples and how one should cope with the resulting uncertainty. The main principle is that uncertainty decreases smoothly(*) with sample size: there is absolutely no magic threshold that separates unreliable from reliable correlations.
With a small sample it is, for example, very likely that a few points, perhaps as few as one, dominate the calculation and we need to know whether the calculated result makes sense.
Regardless of that, it is always a good idea to plot the data to see if the correlation makes sense. The example data you give are indeed illustrative of a weak positive correlation. I would not call any data point an outlier, nor do I want to call the pattern nonlinear, although as always part of the problem is that the sample size is small. I confirm Pearson correlation of $0.24$ and add Spearman correlation of $0.23$.

Note a small clash of terminology: "number of samples" (terminology in several scientific fields, especially when the samples are physical, soil, sediment, water, blood, whatever) is for statistically-minded people "sample size". That is, in statistical terms, you have one sample with 6 observations here.
(*) The wording "smoothly" won't satisfy those with a taste or talent for mathematical rigour. Sample size clearly changes discretely. The key point is that there is no threshold.