I'd agree with the first definition in relation to the meaning of the word "parameter" in statistics.
As it says, it's some population characteristic. So for example, a population mean or a population tenth percentile, population maximum, population interquartile range, population correlation, population second Pearson skewness, population coefficient of variation would all be examples of parameters.
Where you have a population distribution specified by some finite set of parameters (such as $\mu$ and $\sigma$ in the normal), these population parameters will be defined in terms of those distribution parameters. So in a normal ($N(\mu,\sigma^2)$) the population interquartile range is about $1.349\sigma$, but in an exponential with rate parameter $\lambda$ the population interquartile range is about $1.0986/\lambda$.
An example of a parameter that's not a regression coefficient can be found even in a regression -- the obvious one would be the population variance of the error term ($\sigma^2$).
Population regression coefficients (of which the sample regression coefficients you obtain by fitting a regression model are intended to be estimates) are certainly examples of parameters, but parameters needn't be the coefficients in a regression. They can be pretty much any numerical characteristic of a population.
Another example of a parameter would be the shift parameter of a shifted exponential distribution; there was a recent question here where a regression model with an additive exponential noise term was being used (where interest focused on estimating a population line which determined the shift parameter -- that is, to estimate the line defining the lower edge of where the data could be).
You can also have parameters for the mean in a nonlinear regression; at least some of them must enter the expectation non-linearly.
An example would be the Michaelis–Menten model (often used for modelling the rate of enzymatic reactions), which can be written
$E(Y|x) = \frac{\lambda}{1+\kappa/x} = \frac{\lambda\, x}{\kappa+x}$
where $Y$ is typically a random variable representing a reaction rate and $x$ is the concentration of a substrate; $\lambda$ and $\kappa$ are parameters (as is the variance of the implied noise term not shown here) representing the maximum rate at saturation and the substrate concentration required to achieve half the maximum rate respectively. Even though it's a parameter defining the conditional mean of $Y$ (i.e. defining the "regression"), the $\kappa$ parameter certainly doesn't enter this regression model linearly.