0
$\begingroup$

I have access to a black-box sampler (a neural network trained by someone else). The sampler takes as an input a discrete $m \in \{1,2,\dotsc,10\}$ and produces a continuous $5$-dimensional random $Q \in \mathbb R^5$ drawn from $P(Q; m)$. I don't know much about $P(Q; m)$, but it seems to be a "good" continuous distribution for each value of $m$.

By executing the sampler with different inputs many times, I can generate a sample $(m_1, q_1), \dotsc, (m_n, q_n)$, where each $q_i$ is drawn (by the aforementioned black-box sampler) from $P(Q; m_i)$. I can control how $m_i$'s are generated (in particular, make them uniform over $\{1,2,\dotsc,10\})$. From this sample, I would like to construct a maximum likelihood estimator (MLE) for the parameter $m$ so that I can further produce estimates $\hat m$ from $Q$. How to do that?

I have two initial ideas, feel free to comment on them.

  1. Approximate $Q$ by a discrete variable. In particular, split $\mathbb R^5$ into non-overlapping $k$ regions $\mathbb R^5 = V_1 \cup V_2 \cup \dotsb \cup V_k$ and transform the sample $(m_i, q_i)$ into the sample $(m_i, j_i)$ such that $q_i \in V_{j_i}$ (i.e., register which of the regions $Q$ belongs to). Then estimate probabilities $P(Q \in V_j | m)$ and construct MLE via arg max decision rule. (I.e., for each new $Q$, I check which region it belongs to, and output $\hat m$ that maximises the likelihood of getting into the region.)
  2. Assume that $P(Q;m)$ can be approximated well enough by some rich parametric distribution and then use MLE of the parameters of the distribution.

In case 1, I am afraid of dimensionality curse, i.e., that I will need a very large $k$ and therefore prohibitevely large size of the sample, $n$.

In case 2, I am not sure which parametric distribution to use so that it is a) rich enough to approximate $P(Q;m)$ well, and b) relatively easy to estimate its parameters.

$\endgroup$
3
  • $\begingroup$ The description of the problem setting seems inconsistent, which makes it difficult even to determine what you are asking. Since $m$ is an "input," in what sense are we to consider it as a "parameter" to be estimated? $\endgroup$
    – whuber
    Nov 9, 2020 at 18:06
  • $\begingroup$ Well, my problem is to identify how much this black box output $Q$ "leaks" about $m$ (assumed $m$ is uniformly distributed). In other words, how good one can guess $m$ if only observes $Q$? $\endgroup$ Nov 9, 2020 at 19:36
  • 1
    $\begingroup$ It sounds like you need a Bayes predictor for $m.$ $\endgroup$
    – whuber
    Nov 9, 2020 at 20:04

1 Answer 1

1
$\begingroup$

Unless I've very much misunderstood, this is a straightforward classification problem: you have five continuous predictors, $Q \in \mathbb{R}^5$, and 10 possible categories, $m \in \{1,2,\dotsc,10\}$, and you can produce as much training data as you like by simulating from the neural network.

You can use almost any classification algorithm for this, and the suitability of different classifiers will depend on the distribution of the features $Q$ within and between categories. Multinomial logistic regression is a good place to start, and this will automatically estimate the probability that each sample belongs to each category.

$\endgroup$
2
  • $\begingroup$ I just wonder how this would relate to MLE? In other words, is there a way to know if, say, multinomial logistic regression the best estimator? $\endgroup$ Nov 9, 2020 at 19:39
  • 1
    $\begingroup$ I'm afraid that question is far too broad to answer here. If I have understood, and this is a straightforward classification problem, then you can find what you're looking for in any introductory textbook (and Elements of Statistical Learning is the one I usually recommend) and create new questions here for anything you're still unsure about. $\endgroup$
    – Eoin
    Nov 9, 2020 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.