In Section "1.1 Monotone Functions" of Chung's classic probability textbook A Course in Probability Theory, it is stated:
How can $f_1$ and $f_2$ differ at all? This can happen only when $f_1(x)$ and $f_2(x)$ assume different values in the interval $(f_1(x-), f_1(x+))= (f_2(x-), f_2(x+))$. It will turn out in Chapter 2 (see in particular Exercise 21 of Sec. 2.2) that the precise value of $f$ at a jump is quite unessential for our purposes and may be modified, subject to (3), to suit our convenience. More precisely, given the function $f$, we can define a new function $\tilde{f}$ in several different ways, such as
\begin{align}
\tilde{f}(x) = f(x-), \tilde{f}(x) = f(x+), \tilde{f}(x) = \frac{f(x-) + f(x+)}{2},
\end{align}
and use one of these instead of the original one. The third modification is found to be convenient in Fourier analysis, but either one of the first two is more suitable for probability theory. We have a free choice between them and we shall choose the second, namely, right continuity.
You can interpret $f_1(x), f_2(x)$ in the above quote as $P(X < x), P(X \leq x)$ respectively (in the original text, $f_1$ and $f_2$ are two nondecreasing functions agreed on a dense set $D \subset (-\infty, +\infty)$, which are discussed for introducing the probability distribution function aftermath). And as the highlighted text shows, in probability theory, there is no particular reason of preferring $f_2(x)$ to $f_1(x)$ (or vice versa). The cited "Exercise 21 of Sec 2.2" basically stated that, with some obvious minor modifications on the relationship between $f_2$ and $\mu$, the fundamental theorem
Each distribution function $f_2$ determines a probability measure $\mu$ on $\mathscr{B}^1$.
still holds for $f_1$ and $\mu$, supporting the highlighted statement above.
To conclude, my short answer to your question is "yes, it is just a matter of taste" -- I remember that my college probability professor, who learned probability theory from Soviet Union's (where the father of modern probability theory A. N. Kolmogorov is from) literature, told us that in Russian probability texts (e.g., the reference mentioned by @User1865345 in the comment), $F(x)$ is defined as $P(X < x)$. On the other hand, in most classical English probability texts that I have seen, such as
- A Course in Probability Theory by Kai Lai Chung
- Probability and Measure by Patrick Billingsley
- Probability: Theory and Examples by Rick Durrett
- Probability with Martingales by David Williams
$F(x)$ is defined as $P(X \leq x)$. One exception is Probability Theory: Independence, Interchangeability, Martingales by Yuan Shih Chow and Henry Teicher, where $F(x)$ is defined as $P[X < x]$.