Diversity in Recommendation Systems

Welcome to today's lesson on diversity in recommendation systems. In our previous lesson, we explored coverage and novelty metrics. Now, we will dive into diversity, an equally important concept that is crucial in enhancing user satisfaction and engagement with recommendation systems. By ensuring that users receive a diverse range of recommendations, we maintain their interest and cater to varied tastes, which ultimately leads to a richer user experience.

Before we dive into the code, let’s quickly ensure we have the necessary setup in place. For this lesson, we need user predictions and item vectors. As a reminder, here’s a brief setup using a simple dictionary for predictions and item vectors.

import numpy as np

# Example user predictions: each user receives a list of recommended items
user_predictions = {
    'user1': ['item1', 'item2', 'item3'],
    'user2': ['item2', 'item3', 'item4'],
    'user3': ['item1', 'item4', 'item5']
}

# Example item vectors representing characteristics of items in a multi-dimensional space
item_vectors = {
    'item1': np.array([1, 0, 0]),
    'item2': np.array([0, 1, 0]),
    'item3': np.array([0, 0, 1]),
    'item4': np.array([1, 1, 0]),
    'item5': np.array([0, 1, 1]),
}

These data structures are essential for calculating diversity and should be loaded into your environment beforehand.

As a reminder, Cosine similarity is a measure used to determine the similarity between two non-zero vectors. In recommendation systems, it helps to measure how similar or diverse the recommended items are based on their vectors. A cosine similarity of 1 means the vectors are identical, while a value of 0 indicates complete dissimilarity.

For two item vectors, A and B, the cosine similarity is calculated as:

$\text{Cosine Similarity}(A, B) = \frac{A \cdot B}{||A|| \times ||B||}$

Where:

• $A \cdot B$ is the dot product of the vectors.
• $||A||$ and $||B||$ are the magnitudes of the vectors.

Understanding this concept is crucial as it is the foundation for calculating diversity.

Let's break down the diversity function and understand its components. First, we process each user's list of recommended items to transform them into vectors using the item_vectors dictionary:

def diversity(predictions, item_vectors):
    # Convert item recommendations to vectors
    item_indices = [
        [item_vectors[item] for item in items if item in item_vectors] 
        for items in predictions.values()
    ]