Return to Question

EDIT

For two signals or random variables to be independent, the mutual information (MI) must be zero. Let, $error = X -Y$ where X is the desired signal and Y is the measured signal. It is desired that error tend to zero. The paper Minimum-entropy estimation in semi-parametric models http://dl.acm.org/citation.cfm?id=1195853 says that minimizing error entropy is equivalent to minimizing mutual information between error and observation. Based on this I have few questions answers to which I am struggling to seek and shall be grateful for intuitive answers

1. What does mutual information (MI) convey? If 2 signals are independent then MI is zero; What does this imply and mean in the case of mutual information of error, and in general definition of MI and entropy. What is the interpretation if MI of error is decreasing or getting minimized?

2. Can somebody please explain What is the physical meaning of mutual information and what is the implication if mutual information between two signals increases and when it decreases?

3. What is the meaning of independence? Does it mean different , unrelated?

EDIT

For two signals or random variables to be independent, the mutual information (MI) must be zero.

1. What does mutual information (MI) convey?
2. What is the meaning of independence? Does it mean different , unrelated?

EDIT

For two signals or random variables to be independent, the mutual information (MI) must be zero. Let, $error = X -Y$ where X is the desired signal and Y is the measured signal. It is desired that error tend to zero.

1. What is the interpretation if MI of error is decreasing or getting minimized?

2. Can somebody please explain What is the physical meaning of mutual information and what is the implication if mutual information between two signals increases and when it decreases?

3. What is the meaning of independence? Does it mean unrelated?

EDIT

For two signals or random variables to be independent, the mutual information (MI) must be zero. Let, $error = X -Y$ where X is the desired signal and Y is the measured signal. It is desired that error tend to zero. The paper Minimum-entropy estimation in semi-parametric models http://dl.acm.org/citation.cfm?id=1195853 says that minimizing error entropy is equivalent to minimizing mutual information between error and observation. Based on this I have few questions answers to which I am struggling to seek and shall be grateful for intuitive answers

1. What does mutual information (MI) convey? If 2 signals are independent then MI is zero; What does this imply and mean in the case of mutual information of error, and in general definition of MI and entropy. What is the interpretation if MI of error is decreasing or getting minimized?

2. Can somebody please explain What is the physical meaning of mutual information and what is the implication if mutual information between two signals increases and when it decreases?

3. What is the meaning of independence? Does it mean different , unrelated?

For two signals or random variables to be uncorrelated, the mutual information (MI) and correlation coefficient must be zero. Let, $error = X -Y$ where X is the desired signal and Y is the measured signal. It is desired that error tend to zero.

1. How does MI play a role if I say that MI of error must decrease?

2. Can somebody please explain What is the physical meaning of mutual information and what is the implication if mutual information between two signals increases and when it decreases?

3. What is the difference then between MI and correlation in this aspect ?

EDIT

For two signals or random variables to be independent, the mutual information (MI) must be zero. Let, $error = X -Y$ where X is the desired signal and Y is the measured signal. It is desired that error tend to zero.

1. What is the interpretation if MI of error is decreasing or getting minimized?

2. Can somebody please explain What is the physical meaning of mutual information and what is the implication if mutual information between two signals increases and when it decreases?

3. What is the meaning of independence? Does it mean unrelated?