I am trying to enhance the learning capabilities of my regression task with the help of active learning. I decided to use variance reduction using the Fisher information matrix as a query strategy as explained in Settles' popular survey "Active learning literature survey, 2010" (https://www.cs.cmu.edu/~10701/slides/Settles_notes.pdf). According to it the output variance of an input x can be approximated by

enter image description here

where F is the Fisher information matrix.

It appears like in this equation y is expected to be a scalar output. I am working with a neural network that has 3 output nodes and I am not sure if the equation can still be applied without modification. Of course I would get a covariance matrix instead of a scalar variance.

Unfortunately I could not find any literature regarding active learning using the Fisher information matrix for regression tasks with multiple outputs.

Can I still use this equation for estimating the output variance of my inputs? Is there any literature that goes into more detail regarding my problem?

  • $\begingroup$ I believe you're correct in your remarks about a covariance matrix. Reasoning purely from linear algebra, a matrix of partial derivatives may replace the left and right factors in the middle portion. Caveat: I have not looked into the details of your application, I'm just pointing out how thinking about matrix products can allow you to generalize. $\endgroup$
    – Sycorax
    Feb 16 '21 at 0:39
  • $\begingroup$ Thanks! That gives me more confidence in the results I'm getting! $\endgroup$
    – Stacksatty
    Feb 16 '21 at 7:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.