In a number of applications involving classification, the final goal is not determining
which class (or classes) individual unlabelled instances belong to, but estimating
the prevalence (or "relative frequency", or "prior probability") of each class in the
unlabelled data. In recent years it has been pointed out that, in these cases, it would
make sense to directly optimise machine learning algorithms for this goal, rather
than (somehow indirectly) just optimising the classifier's ability to label individual
instances. The task of training estimators of class prevalence via supervised learning
is known as learning to quantify, or, more simply, quantification. It is by now well
known that performing quantification by classifying each unlabelled instance via a
standard classifier and then counting the instances that have been assigned to the
class (the Classify and Count method) usually leads to biased estimators of class
prevalence, i.e., to poor quantification accuracy; as a result, methods (and evaluation
measures) that address quantification as a task in its own right have been developed.
This book covers the main applications of quantification, the main methods that
have been developed for learning to quantify, the measures that have been adopted
for evaluating it, and the challenges that still need to be addressed by future research.
The book is divided in seven chapters. Chapter 1 sets the stage for the rest
of the book by introducing fundamental notions such as class distributions, their
estimation, and dataset shift, by arguing for the suboptimality of using classification
techniques for performing this estimation, and by discussing why learning to
quantify has evolved as a task of its own, rather than remaining a by-product of
classification. Chapter 2 provides the motivation for what is to come by describing
the applications that quantification has been put at, ranging from improving classification
accuracy in domain adaptation, to measuring and improving the fairness
of classification systems with respect to a sensitive attribute, to supporting research
and development in the social sciences, in political science, epidemiology, market
research, and others. In Chapter 3 we move on to discuss the experimental evaluation
of quantification systems; we look at evaluation measures for the various types
of quantification systems (binary, single-label multiclass, multi-label multiclass,
ordinal), but also at evaluation protocols for quantification, that essentially consist
in ways to extract multiple testing samples for use in quantification evaluation