In supervised learning, we refer to the regressors as independent variables and response variables as dependent, but from a probabilistic standpoint, I am having trouble understanding this.
To breakdown my confusion, I think it makes sense to consider two separate cases (1) regressors are fixed / constant / deterministic (2) Regressors are random variables
Constants can also be viewed as random variables. We know from probability theory that a constant random variable is independent of any other random variable and we also know that independence is symmetric. So if $X$ is independent of $Y$, then $Y$ is independent of $X$. You can see this easily from conditional probability $P(X,Y) = P(X|Y)P(Y) = P(Y|X)P(Y)$. So if $X$ is independent of $Y$, then we have $P(X|Y) = P(X)$. So $P(Y|X)$ must be $P(Y)$.
But how does this make sense in the context of supervised learning? We assume that $Y$ is dependent on $X$, but not vice versa?
The same idea holds as the above except $X$ is no longer fixed here.