Choosing Clusters and Centroids

Greetings! Our journey into K-means clustering deepens as we explore two crucial elements: the selection of the number of clusters and the initialization of centroids. Our aim is to comprehend these aspects and put them into action using R. Let's move forward!

The K in K-means signifies the number of clusters. Centroids, the centers of each cluster, are equally significant. Their initial placement in K-means is crucial. Poorly initialized centroids can lead to suboptimal clustering — which is why multiple runs with different initial placements are essential. This highlights the importance of choosing both the number of clusters and their initial centroids.

R’s built-in kmeans() function allows us to specify the number of clusters and also to set the initial centroids manually. Let’s see how we can do this in R using ggplot2 for visualization.

library(ggplot2)

# Load the iris dataset
data(iris)
iris_data <- as.matrix(iris[, 1:4])

# Set the number of clusters
num_clusters <- 3

# Randomly select initial centroids from the data
set.seed(42)
initial_centroids_idx <- sample(1:nrow(iris_data), num_clusters)
initial_centroids <- iris_data[initial_centroids_idx, ]

# Run kmeans with user-specified initial centers
kmeans_result <- kmeans(iris_data, centers = initial_centroids, iter.max = 100, nstart = 1)

# Extract cluster assignments and centroids
labels <- as.factor(kmeans_result$cluster)
centroids <- as.data.frame(kmeans_result$centers)

# Prepare data for ggplot
iris_plot <- as.data.frame(iris_data)
iris_plot$Cluster <- labels

# Visualization (using first two features for simplicity)
p_iris <- ggplot(iris_plot, aes(x = Sepal.Length, y = Sepal.Width, color = Cluster)) +
  geom_point(size = 2) +
  geom_point(data = centroids, aes(x = Sepal.Length, y = Sepal.Width), 
             color = "red", shape = 4, size = 5, stroke = 2) +
  labs(title = "K-means Clustering", x = "Sepal.Length", y = "Sepal.Width") +
  theme_bw()

The iter.max parameter in the kmeans() function controls the maximum number of iterations the algorithm will perform before stopping. In the example above, we set iter.max = 100 to allow the algorithm up to 100 iterations to converge. Typically, K-means converges much sooner, but setting a higher value ensures the algorithm has enough opportunity to find stable clusters, especially for more complex datasets. If the algorithm converges before reaching this limit, it will stop early. If you set iter.max too low, the algorithm might stop before finding a good solution; if you set it very high, it may just take a bit longer to run but won't affect the final result once convergence is reached.

Output:

In the code above, centers is set to our chosen initial centroids, and nstart = 1 ensures that only this initialization is used. The resulting plot (assigned to p_iris) shows the data points colored by cluster, with the centroids marked in red.

As we've seen, different initial centroids and different choices for the number of clusters can lead to different results. R’s kmeans() function uses random initialization by default, which means the starting positions of the centroids are chosen randomly from the data. To reduce the risk of poor clustering due to unlucky initialization, you can use the nstart parameter to run the algorithm multiple times with different random initializations and select the best result. This helps mitigate the impact of poor initial centroid placement.

In this lesson, we explored the principles of choosing the number of clusters and initializing centroids in K-means, all within the R environment. You learned how to set the number of clusters, specify initial centroids, and visualize the results using ggplot2. You also saw how different choices can affect the outcome of clustering. Practice these concepts to deepen your understanding and master K-means clustering in R.