Can two events be strongly or weakly dependent? I've heard correlation for two variables or two random variables (strong or weak dependence). But do we have such concepts for events? Can two events, such as rainy weather and cloudy sky be correlated? Can they be weakly or strongly dependent? In each case how can one determine the degree of dependence?
 A: Covariance and correlation that is just its normalized version, measure the linear relationship between two variables. A linear relationship is just one of the many possible relationships, so they cannot be thought of as a general measure of how strongly the two variables are related, though it's a popular proxy for measuring it.
If you have two events $X$ and $Y$, the most basic way of measuring how strongly they co-occur, is to look at their joint probability $P(X \cap Y)$, the probability of observing $X$ and $Y$ together. It's easy to interpret, since the probability is bounded between zero and one, so the closer to one, the stronger is the relationship.
If the events were independent of each other, the probability would be $P(X \cap Y) = P(X)P(Y)$ by definition. So joint probability higher than this means that the events are more likely to occur together than if they were independent, if lower, less likely as compared to case if they were independent. That’s why you can see the $\tfrac{P(X \cap Y) }{P(X)P(Y)}$ part in metrics like in mutual information or pointwise mutual information.
A: Pointwsei mutual information is a measure of association between two single events.
$$\operatorname{pmi}(x;y) \equiv \log\frac{p(x,y)}{p(x)p(y)} = \log\frac{p(x|y)}{p(x)} = \log\frac{p(y|x)}{p(y)}$$
As you can see, it compares the conditional probability of $P(x|y)$ to $P(x)$, the probability of occurring $x$. It also compares their joint probability with their marginalized probability under the independence assumption.
It can be positive, like pmi of rainy weather and cloudy sky, which implies that $P(rainy\ weather | cloudy\ sky)$ is more than $p(rainy\ weather)$ without any condition. It means that rainy weather tends to co-occur with cloudy sky. However pmi is symmetric, and it can be said about cloudy sky conditioned on rainy weather.
It can be zero, when they are independent, like rainy weather and the probability of my garbage being collected today ( I found the example online, don't we have better examples?)
And it can be even negative, when they tend not to co-occur like rainy weather and clear sky. The positive and negative dependence could be again weak or strong depends on how positive or negative they are.
