8
$\begingroup$

I find references to transformed values in the Stan Reference and User Guides, and example code but no clear tutorial explanation. I'd be grateful for a link.

Michael Betancourt, in his Stan Modeling Language lecture, says this:

"The transformed parameters block allows for parameter processing before the posterior is computed"

And he offers this example:

transformed parameters {
    real<lower=0> lambda;
    lambda <- lambda1 + lambda2;
}

Andrew Gelman in his Intro to Bayesian Data Analysis and Stan, offers this example:

parameters {
    real b;
    real<lower=0> sigma_a;
    real<lower=0> sigma_y;
    vector[nteams] eta_a;
}
transformed parameters {
    vector[nteams] eta_a;
    a = b*prior_scores + sigma_a*eta_a; 
}
model {
    eta_a ~ normal(0,1);
    sqrt_dif ~ student_t(df, a[team1] - a[team2], sigma_y);
}

At times, lecturers talk of the transformed parameters block as though it were nothing but initialization code run at the beginning of the fitting process, but at other times, they talk of it in ways more appropriate for its hi-falutin title of "transformed" parameters, as if this were some kind of kernel or other transformation that makes the calculations more tractable or that maps them into, say, a linear space, or that constrains them to manageable intervals in ways similar to a logit() or exp().

What's the theoretical vision or purpose behind a special section for "transformed" parameters?

$\endgroup$
  • $\begingroup$ are you asking for how transformed variables are implemented in this particular software package, or are you asking why in general parameter transformations are useful for posterior sampling? $\endgroup$ – Taylor Nov 7 '18 at 4:36
  • $\begingroup$ Both. :-) Anything you care to offer in either area would be helpful. $\endgroup$ – John Strong Nov 7 '18 at 13:28
  • 1
    $\begingroup$ I don't use Stan, but I do this when I want my posterior to be more easily approximated by a normal distribution, or when I want to facilitate my choice of proposal distributions in the sampler. If you sample the unconstrained parameter with say, a normal distribution, then you can transform the sample back. If you need to evaluate this density, too, in addition to sampling from it, you just need to be able to evaluate normal densities and figure out jacobians. I suspect Stan has code to automatically handle the jacobians for you. $\endgroup$ – Taylor Nov 7 '18 at 13:51
  • $\begingroup$ That's very helpful, thanks! Would you mind giving me an example? $\endgroup$ – John Strong Nov 7 '18 at 13:56
  • $\begingroup$ The role of Jacobians in this is another thing I'm trying to get a grip on. $\endgroup$ – John Strong Nov 7 '18 at 13:57
8
$\begingroup$

Objects declared in the transformed parameters block of a Stan program are:

  1. Unknown but are known given the values of the objects in the parameters block
  2. Saved in the output and hence should be of interest to the researcher
  3. Are usually the arguments to the log-likelihood function that is evaluated in the model block, although in hierarchical models the line between the prior and the likelihood can be drawn in multiple ways

(if the third point is not the case, the object should usually be declared in the generated quantities block of a Stan program)

The purpose of declaring such things in the transformed parameters block rather than the parameters block is often to obtain more efficient sampling from the posterior distribution. If there is a posterior PDF $f\left(\left.\boldsymbol{\theta}\right|\mbox{data}\right)$, then for any bijective transformation from $\boldsymbol{\alpha}$ to $\boldsymbol{\theta}$, the posterior PDF of $\boldsymbol{\alpha}$ is simply $f\left(\left.\boldsymbol{\theta}\left(\boldsymbol{\alpha}\right)\right|\mbox{data}\right)\mathrm{abs}\left|\mathbf{J}\right|$, where $\left|\mathbf{J}\right|$ is the determinant of the Jacobian matrix of the transformation from $\boldsymbol{\alpha}$ to $\boldsymbol{\theta}$. Thus, you can make the same inferences about (functions of) $\boldsymbol{\theta}$ either by drawing from the posterior whose PDF is $f\left(\left.\boldsymbol{\theta}\right|\mbox{data}\right)$ where $\boldsymbol{\theta}$ are the parameters or the posterior whose PDF is $f\left(\left.\boldsymbol{\theta}\left(\boldsymbol{\alpha}\right)\right|\mbox{data}\right)\mathrm{abs}\left|\mathbf{J}\right|$ where $\boldsymbol{\alpha}$ are parameters and $\boldsymbol{\theta}$ are transformed parameters. Since the posterior inferences about (functions of) $\boldsymbol{\theta}$ are the same, you are free to choose a transformation that enhances the efficiency of the sampling by making $\boldsymbol{\alpha}$ less correlated, unit scaled, more Gaussian, etc. than is $\boldsymbol{\theta}$.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Fantastic answer. Would Neal's Funnel be a good example of how to use the transformed parameters block? mc-stan.org/docs/2_18/stan-users-guide/… $\endgroup$ – eric_kernfeld Feb 25 at 15:52
  • 2
    $\begingroup$ @eric_kernfeld Yes, the non-centered version of Neal's funnel is a an example of using the transformed parameters block appropriately. But there are no shortage of examples. When I write a Stan program, I usually am putting the substantively important unknowns into the transformed parameters block and then figuring out the best way to configure the parameters block and the transformations to achieve the best sampling. $\endgroup$ – Ben Goodrich Feb 29 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.