With vectors of fixed finite dimension, say $m$, you can do this in various equivalent ways. Write $X_{ni}$ for the components of $X_n$.
- $X_n=O_p(1)$ when the components $X_{ni}$ of $X_n$ are $O_p(1)$ as scalar sequences.
- $X_n=O_p(1)$ when the Euclidean norm $\|X_n\|_2=\sqrt{\sum_{i=1}^m X_{ni}^2}$ is $O_p(1)$ (note: this is the random quantity $\|X_n\|_2$, not the possibly non-existent $E[\|X_n\|_2])$
- Like 2, but for any $p$-norm $\|X_n\|_p=\sqrt[p]{\sum_{i=1}^m |X_{ni}|^p}$, for $0<p\leq \infty$
These are all equivalent to each other, and they are equivalent to tightness of the sequence, which is a topological condition: for any $\epsilon$ there exists a compact set $K_\epsilon$ such that $P(X_n\in K_\epsilon)>1-\epsilon$ for all $n$.
These are all different if the dimension is not finite or is allowed to grow with $n$; in that context you would usually be explicit and say $\|X_n\|_2=O_p(1)$ or $X_{ni}=O_p(1)$ componentwise. In particular, definition 1 is a very weak condition for infinite-dimensional vectors.