VIF calculation in regression

I want to use VIF to check the multicollinearity between some ordinal variables and continuous variables. When I put one variable as dependent and the other as independent, the regression gives one VIF value, and when I exchange these two, then the VIF is different. And once the VIF value is higher than 3, and the other time it is lesser than 3.

Then, how I do make a decision to keep the variable or not, and which one should I keep? Ultimately, I am going to use these variables in a logistic regression. How important it is to see multicollinearity in logistic regression?

VIF is a measure of how much the variance of the estimated regression coefficient $b_k$ is "inflated" by the existence of correlation among the predictor variables in the model. A VIF of 1 means that there is no correlation among the $k_{th}$ predictor and the remaining predictor variables, and hence the variance of $b_k$ is not inflated at all. The general rule of thumb is that VIFs exceeding 4 warrant further investigation, while VIFs exceeding 10 are signs of serious multicollinearity requiring correction.