How to group/cluster this information about consumers? I have several millions of rows similar to those below, describing consumers(data has been modified):




city
road
house_number
postcode
state




fort worth
sycamore school rd
5421
76123
texas


fort worth
sycamore school road
4750
76133
tx




I want to group/cluster this data, so that people who live closer stay in the same group/cluster.
My idea was to join in cell by row, creating a string, e.g. for row 1 we would have "fort worth sycamore school rd 5421 76123 texas" and then use a similarity deep learning similarity model to vectorize each sentence/row, and then apply a clustering method to those vectors.
However, I found that deep learning similarity models usually use high-dimensional embeddings of 300 dim or more, and that for a clustering method seems to be highly prohibitive... Also, the similarity models that I found weren't capable of understanding that "tx" is "texas".
What would you advise me to do?
 A: I recommend against attempting to one-hot-encode this data, or any machine learning embedding, because your data can already be mapped to spatial coordinates using geospatial databases. Rather, I would first look at mapping these data to latitudes and longitudes.
With the latitude and longitude of each point, distances can be computed as arcs on the surface of a sphere between such points. You can use this to occupy a distance matrix.
With a distance matrix, you could perform distance-based clustering using one of a variety of algorithms. There are a lot of clustering algorithms to choose from that make tradeoffs. The Scikit-Learn package nicely visualizes this for some common algorithms.

The rows of the above grid of diagrams are some easy-to-generate data sets with varying statistical, geometric, and (persistent) homological properties. The same documentation tabulates some of the aspects you might like to consider. Fortunately your (latitude, longitude) values can be readily plotted and colour-coded by the partitioning algorithm which can help you evaluate how sensible the output is.




Method name
Parameters
Scalability
Usecase
Geometry (metric used)




K-Means
number of clusters
Very large n_samples, medium n_clusters with MiniBatch code
General-purpose, even cluster size, flat geometry, not too many clusters, inductive
Distances between points


Affinity propagation
damping, sample preference
Not scalable with n_samples
Many clusters, uneven cluster size, non-flat geometry, inductive
Graph distance (e.g. nearest-neighbor graph)


Mean-shift
bandwidth
Not scalable with n_samples
Many clusters, uneven cluster size, non-flat geometry, inductive
Distances between points


Spectral clustering
number of clusters
Medium n_samples, small n_clusters
Few clusters, even cluster size, non-flat geometry, transductive
Graph distance (e.g. nearest-neighbor graph)


Ward hierarchical clustering
number of clusters or distance threshold
Large n_samples and n_clusters
Many clusters, possibly connectivity constraints, transductive
Distances between points


Agglomerative clustering
number of clusters or distance threshold, linkage type, distance
Large n_samples and n_clusters
Many clusters, possibly connectivity constraints, non Euclidean distances, transductive
Any pairwise distance


DBSCAN
neighborhood size
Very large n_samples, medium n_clusters
Non-flat geometry, uneven cluster sizes, outlier removal, transductive
Distances between nearest points


OPTICS
minimum cluster membership
Very large n_samples, large n_clusters
Non-flat geometry, uneven cluster sizes, variable cluster density, outlier removal, transductive
Distances between points


Gaussian mixtures
many
Not scalable
Flat geometry, good for density estimation, inductive
Mahalanobis distances to  centers


BIRCH
branching factor, threshold, optional global clusterer.
Large n_clusters and n_samples
Large dataset, outlier removal, data reduction, inductive
Euclidean distance between points


Bisecting K-Means
number of clusters
Very large n_samples, medium n_clusters
General-purpose, even cluster size, flat geometry, no empty clusters, inductive, hierarchical
Distances between points




I cannot tell you apriori which of these algorithms will be the most suitable for your data. But hopefully this gives you a sense of some of the algorithms that are out there, and that they are non-equivalent in the partitions they recommend.

Here is an afterthought. The main purpose of hard clustering, as opposed to fuzzy clustering, is to suggest a partition. You can also choose partitions based on other data. For example, you could simply partition your data (spatially) by the city they live in.

Here is another afterthought. If you wanted to cluster by distances, you have some choice in what space you are calculating distances within. Above I recommended using arcs on a sphere. You could also use distances along paths in a transportation network (if that is relevant to your research question).

String mapping such as $\text{tx} \rightarrow \text{texas}$ can be done using traditional programming; no machine learning required.
