0
$\begingroup$

I have a function that outputs a value of $y = 6$ at some datapoint. Now I decrease the dimension of that function and the updated function outputs say, $y = 5.7$. How do I compute information loss as a result of this change in dimensionality reduction? Assuming that the first function has higher dimension and second function has lower, how do I quantify such "relative" information loss by using the outputs from the two functions?

$\endgroup$
13
  • 3
    $\begingroup$ Reposting a question does not reopen it: it just irritates people who have to consolidate or link the two threads. Please read our help center for more about how this site works. $\endgroup$
    – whuber
    Apr 1, 2021 at 1:37
  • 2
    $\begingroup$ The reason was given: "This question needs details or clarity." $\endgroup$ Apr 1, 2021 at 2:26
  • 1
    $\begingroup$ "Model" and "relative information loss" are so vague that both need elaboration. $\endgroup$
    – whuber
    Apr 1, 2021 at 13:57
  • 1
    $\begingroup$ Uncertainty is also defined in terms of a distribution. You’ve mentioned nothing about accuracy so far, because you haven’t provided a ground truth. It’s not clear that accuracy is relevant here. You’ve given two specific model outputs (for the same input). If you want to define information loss, you need to do so according to a distribution. This is true of basically every quantity in information theory. Take some time to think this through, then edit those details into your question. If there are parts of this that confuse you, they should be posted as new, self-contained questions. $\endgroup$ Apr 1, 2021 at 16:22
  • 1
    $\begingroup$ This paper may help you to organize your thoughts: arxiv.org/abs/1204.0429 $\endgroup$ Apr 1, 2021 at 16:29

0

Browse other questions tagged or ask your own question.