Torchmetrics v1.2 is out now! The latest release includes 11 new metrics within a new subdomain: Clustering.
In this blog post, we briefly explain what clustering is, why it’s a useful measure and newly added metrics that can be used with code samples.
Clustering - what is it?
Clustering is an unsupervised learning technique. The term unsupervised here refers to the fact that we do not have ground truth targets as we do in classification. The primary goal of clustering is to discover hidden patterns or structures within data without prior knowledge about the meaning or importance of particular features. Thus, clustering is a form of data exploration compared to supervised learning, where the goal is “just” to predict if a data point belongs to one class.
The key goal of clustering algorithms is to split data into clusters/sets where data points from the same cluster are more similar to each other than any other points from the remaining clusters. Some of the most common and widely used clustering algorithms are K-Means, Hierarchical clustering, and Gaussian Mixture Models (GMM).
An objective quality evaluation/measure is required regardless of the clustering algorithm or internal optimization criterion used. In general, we can divide all clustering metrics into two categories: extrinsic metrics and intrinsic metrics.
Extrinsic metrics
Extrinsic metrics are characterized by requirements of some ground truth labeling, even if used for an unsupervised method. This may seem counter-intuitive at first as we, by clustering definition, do not use such ground truth labeling. However, most clustering algorithms are still developed on datasets with labels available, so these metrics use this fact as an advantage.
Intrinsic metrics
In contrast, intrinsic metrics do not need any ground truth information. These metrics estimate inter-cluster consistency (cohesion of all points assigned to a single set) compared to other clusters (separation). This is often done by comparing the distance in the embedding space.
Update to Mean Average Precision
MeanAveragePrecision, the most widely used metric for object detection in computer vision, now supports two new arguments: average and backend.
-
The
averageargument controls averaging over multiple classes. By the core definition, the default way ismacroaveraging, where the metric is calculated for each class separately and then averaged together. This will continue to be the default in Torchmetrics, but now we also support the settingaverage="micro". Every object under this setting is essentially considered to be the same class, and the returned value is, therefore, calculated simultaneously over all objects. -
The second argument -
backend, is important, as it indicates what computational backend will be used for the internal computations. SinceMeanAveragePrecisionis not a simple metric to compute, and we value the correctness of our metric, we rely on some third-party library to do the internal computations. By default, we rely on users to have the official pycocotools installed, but with the new argument, we will also be supporting other backends.
[1.2.0] - 2023-09-22
Added
- Added metric to cluster package:
MutualInformationScore(#2008)RandScore(#2025)NormalizedMutualInfoScore(#2029)AdjustedRandScore(#2032)CalinskiHarabaszScore(#2036)DunnIndex(#2049)HomogeneityScore(#2053)CompletenessScore(#2053)VMeasureScore(#2053)FowlkesMallowsIndex(#2066)AdjustedMutualInfoScore(#2058)DaviesBouldinScore(#2071)
- Added
backendargument toMeanAveragePrecision(#2034)
Full Changelog: v1.1.0...v1.2.0
New Contributors since v1.1.0
- @matsumotosan made their first contribution in #2008
- @GlavitsBalazs made their first contribution in #2042
- @OmerShubi made their first contribution in #2081
- @munahaf made their first contribution in #2082
Key Contributors
If we forgot someone due to not matching commit email with GitHub account, let us know :]