Mastering K-means Clustering with Python: From Theory to Practical Implementation

Introduction

Welcome to our exploration of Unsupervised Learning and Clustering . In this lesson, we'll delve into K-means clustering, clarify its underlying principles, and navigate through the implementation of the K-means clustering algorithm in Python.

Understanding Unsupervised Learning

Unsupervised Learning uses a dataset without labels to identify inherent patterns. Unlike Supervised Learning, which leverages known outcomes from data for label prediction, Unsupervised Learning operates independently. One application is market basket analysis, which predicts customer purchases based on associated buying behaviors.

K-means Clustering: Theory and Implementation Overview

Let's encapsulate the essence of K-means clustering : this iterative algorithm partitions a group of data points into a predefined number of clusters based on their inherent distances from each other. The K in K-means denotes the number of clusters. K-means clustering operates based on a set metric, the most common of which is the Euclidean distance. In subsequent sections, we'll adopt a hands-on approach to implement K-means clustering in Python. We'll be using libraries like numpy for numerical operations and matplotlib for visualizations. Let's get started!

Initializing and Preparing for K-means Clustering

First, we initiate the lesson by loading the necessary libraries and defining our data points: Python import numpy as np import matplotlib.pyplot as plt np.random.seed(0) x1 = np.random.normal(loc=5, scale=1, size=(100, 2)) x2 = np.random.normal(loc=10, scale=2, size=(100, 2)) x = np.concatenate([x1, x2]) plt.scatter(x[:,0], x[:,1], label='True Position') plt.show() Next, we ready our dataset for K-means clustering. Here, we plot the data points, indicate the number of clusters, and initialize the centroids. Additionally, we introduce helper functions for computing Euclidean distances and assigning centroids. Python k = 3 centroids = x[np.random.choice(range(x.shape[0]), size=k, replace=False), :] def calc_distance(X1, X2): return (sum((X1 - X2)**2))**0.5 def find_closest_centroids(ic, X): assigned_centroid = [] for i in X: distance=[] for j in ic: distance.append(calc_distance(i, j)) assigned_centroid.append(np.argmin(distance)) return assigned_centroid

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

K-means Clustering with sklearn

For applications that require quick prototyping or dealing with large multidimensional datasets, implementing K-means clustering algorithm from scratch may not be feasible. Thankfully, Python provides the Scikit-Learn library, also known as sklearn, which comes with many efficient tools for machine learning and statistical modeling, one of which is the KMeans.

Firstly, we import the necessary libraries and initialize our data:

Python

import numpy as np
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

# Let's define our data
x = np.array([[2, 10],
              [2, 5],
              [8, 4],
              [5, 8],
              [7, 5],
              [6, 4],
              [1, 2],
              [4, 9]])

Let's instantiate a KMeans object and fit the model to our data. Here we set n_clusters as 3, the number of clusters we want. init is set to 'k-means++', this initializes the centroids to be (generally) distant from each other, leading to probably better results than random initialization.

Python

kmeans_model = KMeans(n_clusters=3, init='k-means++')
kmeans_model.fit(x)

After the model is fitted to the data, the labels of the clusters can be obtained by calling kmeans_model.labels_, and the cluster centers (or 'centroids') can be obtained by calling kmeans_model.cluster_centers_.

Python

centroid = kmeans_model.cluster_centers_
labels = kmeans_model.labels_

Now we shall plot the data points, with the colour of the points denoting the clusters they belong to, and the centroids marked in red.

Python

plt.scatter(x[:,0], x[:,1],c=labels)
plt.scatter(centroid[:,0],centroid[:,1],c='red')
plt.show()

This shows our data points now clustered into three distinct clusters. It's important to note that while the sklearn implementation automates many parts of K-means clustering, understanding the underlying processes and principles is crucial in better interpreting the results and troubleshooting, if necessary.

When we use K-means clustering in machine learning, we aim to partition our dataset into 'k' distinct clusters. The algorithm works by randomly initializing points as cluster centers and iteratively refining the cluster assignment and the center points. But there's a catch: the algorithm's success can depend hugely on how those initial centers are chosen. If you get lucky, you could have a good set of starting points, and hence, quicker and better convergence to an optimal solution. If not, the algorithm can converge to a less optimal solution. In machine learning, we're not fans of relying on such luck!

The n_init parameter tells the KMeans algorithm, "Hey, don't just start once; try multiple times with different random initializations and pick the best outcome." Essentially, n_init represents the number of times the algorithm will run with different centroid seeds.

To use n_init, you simply specify it when creating a KMeans instance:

Python

kmeans = KMeans(n_clusters=3, init='k-means++', n_init=10, random_state=42)

Setting random_state at the same time ensures that your 'luck' is reproducible, meaning you can get the same result every time you run the algorithm with that state. It's like your very own space-time anchor in the realm of randomness!

And that, my dear pioneering data astronaut, is how n_init equips you with the power to navigate the stochastic stars of K-means clustering.

Introduction

Understanding Unsupervised Learning

K-means Clustering: Theory and Implementation Overview

Initializing and Preparing for K-means Clustering

Join the 1M+ learners on CodeSignal

New Centroids Calculation

Performing and Visualizing K-means Clustering

K-means Clustering with sklearn

Conclusion and Practice Mentoring