I am trying to find the values $v_1$ and $v_2$ that maximizes the likelihood of some observations.

I have information about $v_1$ and $v_2$ from a set of 'experiments'. In each experiment, $v_1$ and $v_2$ are corrupted with zero mean Gaussian noise, and then compared to each other, and the maximum of the two is reported. So if I have six experiments, I end up with six binary values. The standard deviation of the noise is $\sigma_c$, equal for all experiments.

I also have additional (independent) information about $v_1$ and $v_2$. I observe two samples, $v^o_1$ and $v^{o}_2$, sampled from a Gaussian distribution with mean equal to $v_1$ and $v_2$ (respectively) and standard deviation $\sigma_v$ (equal for the two observations).

The problem is the following. I need to maximize $p(v\mid v^o, \text{experiments}, \sigma_v,\sigma_c)$. Yet this is maximal when $v=v^o$ and $\sigma_v$ goes to zero. This occurs because the probability density goes to infinite at this point. Not sure how to deal with this. Any pointer in the right direction is much appreciated.


If I understand your question correctly, you should maximize the the joint likelihood of all binary outcomes, i.e. written out as a cross-entropy - not a Gaussian - with the predicted probabilities in that cross-entropy derived from the Gaussian.

First, let $r_n\in\{0,1\} $ be your experiment outcomes, with a convention that $r_n=1$ when $v_1^{o_n}>v_2^{o_n}$. These outcomes $r_n$ are generated in Bernoulli trials with probabilities $p_n=p(r_n=1)$. You need to maximize the log likelihood: $$ L = \sum\limits_{n=1}^N ((1-r_n)log(1-p_n)+r_n log(p_n)) $$

$p_n=p_n(v_1,v_2)$ is a function of the means $v_1$, $v_2$ which is calculated from a 2d Gaussian $\mathcal{N(v_1,v_2;\sigma^2)}$, where $\sigma^2=\sigma_v^2+\sigma_c^2$.

Let's parameterize the space where this Gaussian lives as $(V_1,V_2)$. Then a sample $(v_1^{o_n},v_2^{o_n})$ from this distribution satisfies $v_1^{o_n}>v_2^{o_n}$ if it falls anywhere within the half of the coordinate plane where $V_1>V_2$.

The probability of that is $$ p_n=p(v_1^{o_n}>v_2^{o_n})=\iint\limits_{V_1>V_2} \mathcal{N(v_1,v_2;\sigma^2)} dV_1dV_2 $$

you can rewrite this integral in a sequential form, e.g.

$$ p_n=\int\limits_{-\infty}^{+\infty} dV_2\int\limits_{V_2}^{+\infty} \mathcal{N}(v_1,v_2;\sigma^2)dV_1 $$

This is the expression for $p_n=p_n(v_1,v_2)$ that you can plug into the cross-entropy likelihood and numerically find the $v_1$ and $v_2$ that maximize $L$. This should be pretty feasible in a software or language for numerical analysis.

Note that this solution is only valid for independent Bernoulli trials.

Hope this helps!

| cite | improve this answer | |
  • $\begingroup$ Thanks! I think it is not the exact problem though. I edited my question trying to clarify. The observations $\sigma_v$ has no influence on the results of the experiments; it just controls how close the observations $v^o$ are to the corresponding $v$'s. $\endgroup$ – arzyl Dec 24 '18 at 19:39
  • $\begingroup$ So you have $N=6$ binary decision experiments and $N=6$ independent experiments with sampling? In that case you can estimate $v_1$ and $v_2$ from either cross-entropy as above (with $\sigma^2=\sigma_c^2$) or from a Gaussian likelihood with six observations, or you can combine the two e.g. in a sum or a weighted sum of likelihoods to optimize them jointly. $\endgroup$ – thepainofstats Dec 24 '18 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.