What does "parameterized by" mean? Sometimes I have seen likelihood written as $L(\mu,\sigma |y)$ and sometimes as $L(y|\mu,\sigma)$.
I have been told that in the first case it means that there is a pre-assumed model depicting the probability density of $\mu$ and $\sigma$ parameterized by $y$. In the second case it means the pre-assumed model depicts the probability density of $y$, parameterized by $\mu$ and $\sigma$
I have seen the $|$ symbol used before in Bayes theory and read it as "given that"
I am from a programming background (C#) and tend to think of parameters as inputs to a function.
How should I be thinking about this word in statistics?
 A: “Parametrized by” means that the function $f$ of $x$ has additional parameters $\Theta$, so we can write it as $f(x;\Theta)$. In such a case, we call the function by $x$ given a fixed value of $\Theta$.
Likelihood has two meanings, the traditional on, and Bayesian. Traditionally, likelihood is written as
$$
L(\theta|x) \propto P(x|\theta)
$$
The vertical bar $|$ is used on the right-hand side to denote conditional probability and is a slight abuse of notation on the left-hand side. People write it as $L(\theta|x)$ to show that we keep the data $x$ fixed, but we evaluate the function for different parameters $\theta$. In a Bayesian setting, you usually would not see the left-hand side notation, just the right-hand side will be called the likelihood (here you might have seen $L$ used instead of $P$). See Wikipedia entry on likelihood seems ambiguous for more discussion.
A: What is parameter? What is a parametric model?
Definition $1.$ Let $(\Omega,\mathfrak F) $ be a probability space. The set of probability measures $\{ \mathbb P_\theta:~{\boldsymbol \theta\in\Theta}\}$ indexed by a parameter $\boldsymbol\theta$ is a parametric family if and only if $\Theta\in\mathbb R^n,~n\in\mathbb Z^{>0}$, and each probability measure is known when $\boldsymbol\theta$ is known. Here $\Theta$ is parameter space.
Remark $1.$ A parametric model assumes the population comes from a parametric family.
Example $1.$ A parametric family of $n\in\mathbb Z^{>0}$ dimensional normal distributions indexed by $( \boldsymbol\mu, \mathbf \Sigma) $ is
$$\{\mathcal N_n(\boldsymbol \mu,\mathbf \Sigma): \boldsymbol\mu\in\mathbb R^n,\mathbf \Sigma\in\mathcal M_n\}.\tag 1$$
So, basically the probability measure is labeled or indexed by parameter(s) and the primary objective of inference is to draw information about the parameter $\theta$ in order to know the probability measure (if it is $\sigma$-finite, then the set is identified by the densities; so the problem is to infer about the parameter to draw information about the associated pdf).
$\bullet$ $\mathcal L(\boldsymbol\theta|\mathbf y) $ denotes, given that $\mathbf Y= \mathbf y$ is realised or observed,  the likelihood of parameter $\boldsymbol\theta.$

References and further read:
$\rm [I]$ Mathematical Statistics, Jun Shao, Springer Science$+$Business Media, $2003, $ section $2.1.2, $ p. $94.$
$\rm [II]$ Testing Statistical Hypotheses, E. L. Lehmann, Joseph P. Romano, Springer Science$+$Business Media, $2005, $ section $1.1, $ p. $3.$
$\rm [III]$ Statistical Inference, George Casella, Roger L. Berger, Wadsworth, $2002,$ section $6.3, $ p. $290.$
