11
$\begingroup$

I have a logistic regression model: $softmax(WX)$ where $W$ is my parameter matrix and $X$ is my input. I want a density function over the outputs of that model.

Say I know that my $X$ are distributed according to some density $p$. From the change of variables I know that the density of the outputs will be something like $p'(x) = p(x) det|J^{-1}|$ where $J = \frac{\partial{softmax(WX)}}{\partial{X}}$.

However, this holds only for a bijective function $softmax(WX)$.

What can I do if this is not the case - especially if $W$ is not square?

$\endgroup$
1
  • 2
    $\begingroup$ divide the domain of your non-bijective function into parts where the function is bijective and then apply change of variables. This might work. $\endgroup$
    – mpiktas
    Feb 13, 2011 at 21:05

2 Answers 2

3
$\begingroup$

The generalization of the change of variable formula to the non-bijective case is generally hard to write out explicitly, check http://en.wikipedia.org/wiki/Probability_density_function#Multiple_variables which essentially formalizes mpiktas's suggestion

$\endgroup$
2
$\begingroup$

I found this article

https://en.wikibooks.org/wiki/Probability/Transformation_of_Probability_Densities

excellent!

$\endgroup$
1
  • 1
    $\begingroup$ +1 I have been looking for the $\mathbb{R}^n \mapsto \mathbb{R}^m$ case. $\endgroup$
    – Galen
    Oct 5, 2022 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.