I usually hear "solver" used to describe software, meaning it applies to a particular implementation of an algorithm. For example, this seems to apply to most of the SciComp.SE questions tagged solver.
Generally the term seems to be reserved for mathematical problems with a "well-defined solution". A unique solution would qualify as "well defined" sufficient, as noted by Sycorax in the comments. (The Gurobi solvers seem to be for problems in this class; for what its worth, Gurobi looks like a suite or library of solvers to me).
But I do not think unique is necessary. For example both local and global optimization problems can have well-defined but non-unique solutions (e.g. the function $f[x]=\sin[\pi x]^2$ has global minimum $f[k]=0$ for $k\in\mathbb{Z}$).
I disagree with this answer, which seems to be discussing "equation-system solvers" rather than "optimization solvers". For example in linear least squares, the linear algebra problem is over-determined, but the optimization problem is convex, with a unique solution (in non-degenerate cases). Note also that the Wikipedia "solver" page linked in that answer lists "Linear and non-linear optimisation problems, Shortest path problems, Minimum spanning tree problems" among its examples.
In response to the comment, I will clarify what I mean in the "regression" case.
Given a function $F:\mathbb{R}^n\to\mathbb{R}^m$, a solution to the system of equations specified by
$$F[x]=0$$
is a vector $x\in\mathbb{R}^n$ such that all $m$ components of $F[x]$ are zero. Depending on the function $F$, there can be no solutions, a single unique solution, or many solutions (typically infinitely many), depending on the dimension of the null-space of $F$. In the case where $F$ is linear, i.e. $F[x]=Ax-b$ for some $A\in\mathbb{R}^{m\times{n}},b\in\mathbb{R}^m$, then no solution can exist unless $m\leq\mathrm{rank}[A]\leq n$.
On the other hand for a given objective function $E_F:\mathbb{R}^n\to\mathbb{R}$ and feasible set $\Omega_F\subset\mathbb{R}^n$, which depend on $F$, a solution to the optimization problem specified by
$$\epsilon=\min_{x\in\Omega_F}E_F[x]$$
is a vector $x\in\Omega_F$ such that $E_F[x]\leq E_F[y]$ for all $y\in\Omega_F$.
In "least squares" optimization, the function $E_F$ is a sum of squares. The two most common least squares problems are 1)
$$E_F[x]=\|F[x]\|^2 \text{ , } \Omega_F=\mathbb{R}^n$$
where $F$ corresponds to an overdetermined system of equations, and 2)
$$E_F[x]=\|x\|^2 \text{ , } \Omega_F=\{y\in\mathbb{R}^n\ \mid F[y]=0\}$$
where $F$ corresponds to an underdetermined system of equations.
Common linear algebra platforms, such as Matlab, may combine these three distinct mathematical problems "under the hood" in convenience function such as linsolve(). However low-level ("solver") libraries, such as LAPACK, will not.
Two final points of clarification:
A "solver" will typically correspond to a well-defined but abstracted mathematical problem. For example, "statistical inference" or "sucessful prediction" are not such problems. In the language of Computational Science, you verify a solver, you validate a model.
- The ideas of unique/non-unique or exact/approximate are not fully clear cut. Say we focus just on the case of square systems of equations, which should remove most points of contention. It is ubiquitous to talk of "iterative solvers" in this field (e.g. ~600,000 hits on Google Scholar). So the de facto definition of "solver" must include this class of algorithms, which by definition are essentially inexact.
