The warning you have received is not caused simply by correlations between columns (which are not in themselves a problem) but rather by the fact that your model matrix is not of full rank. Basically, your model is over-parametrized.
This is a problem caused by all the dummy variables together. It is not a problem that can be analysed in terms of pairwise correlations alone. When all the columns of the model matrix are considered together, they are in fact perfectly collinear in one particular dimension, otherwise you would not get the warning that you quote.
If you provided more information about your data and glm call we might be able to give specific advice. On the limited information provided, it is impossible to say more.