Evaluating Hierarchical Clustering
Introduction
Welcome to today's discussion on Hierarchical Clustering. We will study its effectiveness using the Silhouette Score, the Davies-Bouldin Index, and Cross-Tabulation Analysis. We will utilize R's powerful packages, such as cluster, factoextra, and dplyr, to equip you with practical and useful skills for evaluating clustering models.
Hierarchical Clustering in R
R provides robust tools for hierarchical clustering through functions like hclust and dist. In this lesson, we will use these functions to perform hierarchical clustering on a small dataset.
Let's start by creating our data and applying hierarchical clustering:
This code applies hierarchical clustering to our dataset. The resulting cluster assignments are stored in cluster_labels.
Silhouette Score
The Silhouette Score offers a measure to evaluate the effectiveness of our clustering. This score gauges how similar a point is to its own cluster compared to other clusters. Higher scores indicate better clustering.
In R, we can use the silhouette function from the cluster package to compute the Silhouette Score:
The output provides a single score showing the effectiveness of our clustering.
Davies-Bouldin Index
The Davies-Bouldin Index evaluates the average similarity between clusters. A lower index value indicates a better model.
In R, we can use the clusterSim package's index.DB function to compute the Davies-Bouldin Index:
The Davies-Bouldin Index thus obtained serves as another measure of our clustering effectiveness.

