# Are there classifiers with infinite number of classes?

I am writing something about time series classification and wrote the following:

A classifier is a function that for given input $$x_1, ..., x_n$$ (in this case representing data stored in audio file) returns $$y \in C = \{c_1, ..., c_k\}$$, where $$C$$ is a finite set of classes (like $$\{M, F\}$$).

But then I realised that it might not be true. However, I can't think of any example of a classifier with an infinite possible number of classes. Do they exist? If so, what are some examples?

• It is usually called a regression model.
– JTH
Jan 16, 2022 at 16:08
• @JTH Does it mean that we can treat the input of regression as a class? And is it common to think about regression as special case of classification? Most of the times I've seen them in context like "we can divide ML problems into: 1. classification 2. regression 3. maybe something else". It's weird to me to call regression classification but I need to be precise and now I don't know if I should change that definition I wrote or it's acceptable. Jan 16, 2022 at 17:22
• I dispute that infinite-class classification is regression. Doing so ignores the fact that regression has an ordered $y$ variable while classification involves a purely categorical $y$ variable.
– Dave
Jan 16, 2022 at 17:46
• A more productive distinction might be to replace the finite/infinite distinction by "predetermined number" versus "arbitrary number." Obviously the number of classes actually assigned cannot exceed $n$!
– whuber
Jan 18, 2022 at 18:58

According to your definition infinite number of classes should be possible.

One way to think about a classification rule is as a division of feature space into segments. Hence, if we consider a one-dimensional feature space $$x_1 \in \mathbb{R}$$ as an input we can imagine a classifier that returns a class of "$$k$$" for any input within $$[(k-1):k)$$. So it would return "$$1$$" for $$[0:1)$$, "$$2$$" for $$[1:2)$$, etc.

Such a "model" is not estimated from any data but it is a classification rule.

But in practice having a classifier with an infinite number of samples will probably not be possible, because any dataset will be finite. However, we can imagine an "on-line" classifier that is re-adjusted as new samples come in. And in principle new samples can be of a yet-unseen class. Hence the number of classes can keep increasing forever and not be known in advance.

Classifying images to say which integer was depicted would be a classification problem with a countably infinite set of classes, but you would also need an infinitely large image as the input data. It would also be a rather bad way of solving the problem to view it as single classification problem, rather than construct a solution by classifying the digits and constructing the answer from the component parts (if you did it that way, you might not need an infinite dataset to train it).

I doubt there is a practical case of an infinite set of classes as we have rather finite brains and live rather finite lives (in time and space), so we rarely encounter infinities in a meaningful way.

The Distance dependent Chinese restaurant process may be an example. You can think of it as a clustering method, in which the number of clusters is adaptively chosen based on the data. As you add more data points, the number of clusters can in principle increase without bound. So there are an infinite possible number of clusters (or classes), but for any finite amount of data, only finitely many of the clusters are occupied.