Understanding and Implementing Distance Metrics in Hierarchical Clustering

Introduction

Welcome to our lesson on distance metrics in hierarchical clustering! Today, we will delve into the definition and importance of distance metrics, particularly in hierarchical clustering. You will learn about various types of distance metrics such as Euclidean, Manhattan, and Cosine Distance, and how to implement these in Python. After this, we will examine the impact of these distance measures on the resulting hierarchical clustering.

Introduction to Distance Metrics

Distance metrics are essentially measures used in mathematics to calculate the 'distance' between two points. In the context of clustering, we're interested in the distance between data points in our dataset or the distance between clusters of points. We often use metrics like Euclidean distance, Manhattan distance, and Cosine Distance, each with its unique set of characteristics and application scenarios.

Implementing Distance Metrics in Python: Euclidean Distance

The Euclidean distance, often referred to as the straight-line distance between two points in a Euclidean plane, is one of the most commonly used distance metrics in machine learning. Below is the formula and Python implementation of it.

Distance=∑i=1n(pi−qi)2Distance = \sqrt{\sum_{i=1}^{n} (p_i - q_i)^2}

Python code for calculating Euclidean distance:

Python
import math

# Euclidean Distance Metric
def euclidean_distance(point1, point2):
    return math.sqrt(sum((point1 - point2)**2))

Implementing Distance Metrics in Python: Manhattan Distance

The Manhattan distance gets its name from the block-like geographical layout of the Manhattan borough of New York City. The Manhattan distance between two points is the sum of the absolute differences of their coordinates. Here's the formula and Python code for calculating Manhattan distance:

Distance=∑i=1n∣pi−qi∣Distance = \sum_{i=1}^{n} |p_i - q_i|

Python code for calculating Manhattan distance:

Python
# Manhattan Distance Metric
def manhattan_distance(point1, point2):
    return sum(abs(point1 - point2))

Implementing Distance Metrics in Python: Cosine Distance

The third type of distance metric that we will examine today is Cosine distance, but first let's understand the Cosine similarity. Unlike the other two, Cosine similarity measures the cosine of the angle between two vectors, which can be useful in certain multi-dimensional and text classification problems. From that we can calculate the Cosine Distance as 1−Cosine Similarity1 - \text{Cosine Similarity}. Here's the formula and Python function for calculating Cosine Distance:

Cosine Similarity:

Similarity=A⋅B∣∣A∣∣⋅∣∣B∣∣Similarity = \frac{A \cdot B}{||A|| \cdot ||B||}

Cosine Distance:

Distance=1−Similarity=1−A⋅B∣∣A∣∣⋅∣∣B∣∣Distance = 1 - Similarity = 1 - \frac{A \cdot B}{||A|| \cdot ||B||}

Python code for calculating Cosine Distance:

Python
import numpy as np

# Cosine Distance Metric
def cosine_distance(point1, point2):
    return 1 - np.dot(point1, point2) / (np.linalg.norm(point1) * np.linalg.norm(point2))
Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal