Cluster Evaluation with DBSCAN
Introduction
Welcome to our Cluster Performance Unveiled course lesson! Here, we leverage Silhouette Scores, the Davies-Bouldin Index, and Cross-Tabulation Analysis to assess DBSCAN, a top-performing clustering algorithm focused on density. Exciting, right?
Applying DBSCAN and Calculating Silhouette Score
DBSCAN is especially useful when the number of clusters is unknown and density is a key factor in cluster formation. In R, the dbscan package provides a straightforward way to apply the DBSCAN algorithm.
In this implementation, eps specifies the maximum distance between neighboring points, and minPts sets the minimum number of points required to form a dense region (core point). After fitting the algorithm, we need a quantitative measure of clustering quality. The Silhouette Score is a robust indicator, reflecting how similar each point is to its own cluster compared to other clusters.
Calculating the Silhouette Score for DBSCAN Clusters
To calculate the Silhouette Score in R, we use the cluster package:
Why exclude noise points (cluster 0)?
In DBSCAN, points labeled as cluster 0 are considered "noise"—they do not belong to any cluster. Including these noise points in the Silhouette Score calculation would distort the metric, as noise points are, by definition, not assigned to any cluster and may be far from all clusters. This would artificially lower the average Silhouette Score and not accurately reflect the quality of the actual clusters. By excluding noise points, the Silhouette Score more accurately measures the cohesion and separation of the true clusters found by DBSCAN.
A higher Silhouette Score indicates that clusters are dense and well separated.
