When I learned about conditional probability, I found this statement:
if A is not independent of B then also B is not independent of A. Formally, if P(A) ≠ P(A|B) then P(B) ≠ P(B|A).
I think "not independent" is the same as "dependent", right?
So does that mean this statement is also correct: "if A is dependent on B then B is also dependent on A"? I'm a little bit confused because in my mother language, the translation of "dependent" is directed word that's not symmetrical.
In statistics, “dependent” and “not independent” have the same meaning. There is no inherent notion of causation.
In regular English, I would say that “dependent” implies causation. Dinner temperature depends on oven temperature, not the other way around.
Independence is more properly termed mutual independence which eliminates the use of "$A$ is independent of $B$" and replaces it by "$A$ and $B$ are mutually independent". Thus, there is no such thing as $A$ being independent of $B$ and wonderment if that implies that $B$ is dependent of $A$: independence is mutual. Be aware that "$A$ is independent of $B$ if $P(A\mid B) = P(A)$" is an incomplete statement as a definition: $A$ and $B$ can be independent even if $P(A\mid B)$ is undefined e.g. as when $B$ is an event of probability $0$.
The generally accepted definition of independent events is that
$A$ and $B$ are said to be (mutually) independent events if $P(A\cap B) = P(A)P(B)$,
and as in all definitions, the "if" is understood to be "iff" or "if and only if". Note the complete absence of "independent of" and the symmetry in the roles of $A$ and $B$. Except for those who do not believe in the commutativity of multiplication of real numbers or the commutativity of set intersection, the definition works equally well if we interchange $A$ and $B$ throughout in the definition.
Finally, turning to the question of whether "not independent" means "dependent", the answer is Yes.
In probability calculus there is no expression for causal dependency. No one can express with its semantics the popular example that manipulation on barometer do not change the weather, but changes of the weather change barometer measurements. Either two events 'tend to occurs together' (correlate) or not.
The very definition of independence is (probably) derived from the idea, that knowledge if $B$ occurred, do not change probability of occurring event $A$. This is formally written as $P(A) = P(A|B)$.
The contradiction to situation state is lack of independency: the probability of occurring event $B$ increases or decreases probability of occurring event $A$. This is true for barometer and the weather and expressed as $P(A) \neq P(A|B)$.
Mathematicians often know, that their not independency is not always the 'true' dependency and restrain themselves from using causally marked expression. Especially, that in econometrics or causal inference such definition is exist. Therefore at some probability calculus courses you would hear, that no one discussed $dependency$, the discussed ideas were not independency and correlation.
The mathematical tool which analyses dependency in the more natural meaning is do-calculus (by Judea Pearl). This tool extends standard probability calculus with the do operator, which describes intervention in the system. For the barometer and the weather all four statements will be true:
$$P(A) \neq P(A|B)$$ $$P(B) \neq P(B|A)$$ $$P(B) \neq P(B|do(A))$$ $$P(A) = P(A|do(B))$$
In this context I would strongly discourage using word dependent in context of standard probability calculus and statistics. Not independent is good enough, and in fact more precise in context of this 'more advanced' mathematics.
"if A is dependent of B then also B is dependent of A"
Grammatically, that is not correct; the correct preposition is "on", not "of".
Mathematically, the term "dependent" is often used in a nonsymmetric sense: if y is being treated as a being a function of x, then y is dependent on x. In experimental setup, the variable we directly control is called the "independent" variable, and the one that results from the independent variable is called the "dependent" variable.
If you wanted to emphasize the symmetrical nature, you could say "x and y are dependent on each other".
"Not independent" and "dependent" are grammatically same, not only in English but also in other languages as well, including the language of mathematical logic. However, when discussing statistics, one has to use more accurate language. The key observation is that two events can be independent in a unique way, but there are many ways in which they can be dependent (such having a causal relationship, etc.). There is no point in saying that events or random variables are dependent without describing the structure of the dependency. Stating that two events are just "dependent" is meaningless.
