11
$\begingroup$

I have a logistic regression model: $softmax(WX)$ where $W$ is my parameter matrix and $X$ is my input. I want a density function over the outputs of that model.

Say I know that my $X$ are distributed according to some density $p$. From the change of variables I know that the density of the outputs will be something like $p'(x) = p(x) det|J^{-1}|$ where $J = \frac{\partial{softmax(WX)}}{\partial{X}}$.

However, this holds only for a bijective function $softmax(WX)$.

What can I do if this is not the case - especially if $W$ is not square?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ divide the domain of your non-bijective function into parts where the function is bijective and then apply change of variables. This might work. $\endgroup$
    – mpiktas
    Feb 13, 2011 at 21:05

2 Answers 2

Reset to default
3
$\begingroup$

The generalization of the change of variable formula to the non-bijective case is generally hard to write out explicitly, check http://en.wikipedia.org/wiki/Probability_density_function#Multiple_variables which essentially formalizes mpiktas's suggestion

Improve this answer
$\endgroup$
2
$\begingroup$

I found this article

https://en.wikibooks.org/wiki/Probability/Transformation_of_Probability_Densities

excellent!

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ +1 I have been looking for the $\mathbb{R}^n \mapsto \mathbb{R}^m$ case. $\endgroup$
    – Galen
    Oct 5, 2022 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.