A parametric distribution is defined by its family and its parameters. Consider the Poisson family of distributions. This distribution has a single parameter $\lambda$. In GLMs, we wish to model the mean of the parametric distribution as a function of a linear model of covariates. The mean should be called, as you say, or is treated as "the conditional mean (with respect to the covariates)." Therefore, for any parametric distribution, we must reparametrize it such that one parameter represents the mean ($\mu$). Furthermore, each observation is independent with its own mean $\mu_i$, but these means depend on a fixed number of shared parameters: $g(\mu_i) = \beta_0 + \sum_{j=1}^p \beta_p x_{ij}$, with $x_{ij}$ denoting the measurement of covariate $j$ for subject $i$.
For the Poisson family, let $y_i \sim Poi(\lambda_i)$ for $i=1,\cdots,n$ denote a set of independent count data. Suppose we also observe a vector of $p$ covariates $\boldsymbol{x}_i$ for each count datum. We believe that the logarithm of the mean (the canonical link) is linear related to these covariates (i.e. $\log(\mu_i) = \beta_0 + \sum_{j=1}^p \beta_p x_{ij}$). Since the mean of $y_i$, $\mu_i=\lambda_i$, we have that $y_i \sim Poi\left(\lambda_i = \exp \left( \beta_0 + \sum_{j=1}^p \beta_p x_{ij} \right)\right)$. Thus we merely replace $\lambda_i$ in the Likelihood with the model we think is correct.