A. Kazakeviciute, V. Kazakevicius and M. Olivo, "Conditions for Existence of Uniformly Consistent Classifiers," in IEEE Transactions on Information Theory, vol. 63, no. 6, pp. 3425-3432, June 2017. doi: 10.1109/TIT.2017.2696961
Abstract:
We consider the statistical problem of binary classification, which means attaching a random observation X from a separable metric space E to one of the two classes, 0 or 1. We prove that the consistent estimation of conditional probability p(X) = P(Y = 1 | X), where Y is the true class of X, is equivalent to the consistency of a class of empirical classifiers. We then investigate for what classes P there exist an estimate p̂ that is consistent uniformly in p ∈ P. We show that this holds if and only if P is a totally bounded subset of L 1 (E, μ), where μ is the distribution of X. In the case, where E is countable, we give a complete characterization of classes Π, allowing consistent estimation of p, uniform in (μ, p) ∈ Π.