Introduction to Pearson Correlation for Measuring Similarity in Recommendation Systems with JavaScript

Introduction to Similarity Measures in Recommendation Systems

In the world of recommendation systems, one of the keys to success is understanding the similarity between users or items. This understanding forms the backbone of making accurate recommendations. Similarity measures allow us to identify users with similar preferences, improving the quality and relevance of recommendations. In this lesson, we will explore Pearson Correlation, a tool used to measure similarity based on patterns in ratings. By the end, you will be able to implement this measure and understand its application in recommendation systems.

Recap of Essential Setup Steps

Before we dive in, let's quickly recap the setup from previous lessons. We will be using JavaScript and mathjs for this lesson. If you've done these steps before, consider this a helpful reminder. Here's a simple code block to demonstrate setting up and using mathjs to create user rating datasets: JavaScript import { create, all } from 'mathjs'; const math = create(all); // Example user ratings const user1Ratings = math.matrix([5, 3, 4, 2, 1, 5, 3, 4, 2, 1, 5, 3, 4, 2, 1]); const user2Ratings = math.matrix([5, 3, 4, 5, 1, 3, 3, 4, 2, 1, 5, 2, 2, 2, 1]); Each index in user1Ratings and user2Ratings corresponds to the rating of the same item by both users. These arrays can be extracted from the user-item matrix, but this time we will simply define them like this for brevity. If a rating is missing for one user in the user-item matrix, that item should be excluded from the calculation. This ensures that only ratings for items both users have rated are compared.

Understanding Pearson Correlation

r = \frac{\sum (x_i - \bar{x}) (y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2 \sum (y_i - \bar{y})^2}}

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Step-by-step Implementation

Let’s break down the implementation of the Pearson Correlation function to understand how to calculate it step-by-step.

JavaScript

import { create, all } from 'mathjs';
const math = create(all);

// Function to calculate Pearson correlation between two users
function pearsonCorrelation(ratings1, ratings2) {
    const n = ratings1.size()[0];
    if (n !== ratings2.size()[0]) {
        throw new Error("Both arrays must have the same length");
    }

    // Calculate means
    const mean1 = math.mean(ratings1);
    const mean2 = math.mean(ratings2);

    // Calculate the difference from the mean
    const diff1 = math.subtract(ratings1, mean1);
    const diff2 = math.subtract(ratings2, mean2);

    // Calculate numerator and denominator
    const numerator = math.sum(math.dotMultiply(diff1, diff2));
    const denominator = math.sqrt(
        math.sum(math.dotPow(diff1, 2)) * math.sum(math.dotPow(diff2, 2))
    );

    if (denominator === 0) {
        return 0; // Prevent division by zero
    } else {
        return numerator / denominator;
    }
}

import { create, all } from 'mathjs';
const math = create(all);

// Function to calculate Pearson correlation between two users
function pearsonCorrelation(ratings1, ratings2) {
    const n = ratings1.size()[0];
    if (n !== ratings2.size()[0]) {
        throw new Error("Both arrays must have the same length");
    }

    // Calculate means
    const mean1 = math.mean(ratings1);
    const mean2 = math.mean(ratings2);

    // Calculate the difference from the mean
    const diff1 = math.subtract(ratings1, mean1);
    const diff2 = math.subtract(ratings2, mean2);

    // Calculate numerator and denominator
    const numerator = math.sum(math.dotMultiply(diff1, diff2));
    const denominator = math.sqrt(
        math.sum(math.dotPow(diff1, 2)) * math.sum(math.dotPow(diff2, 2))
    );

    if (denominator === 0) {
        return 0; // Prevent division by zero
    } else {
        return numerator / denominator;
    }
}

Calculate Means: We compute the mean rating for each user to understand their general preference level.
Difference from Mean: We find how each rating deviates from the mean.
Numerator and Denominator: We calculate these using the differences to determine the correlation.

Understanding Correlation with Additional Example

The term 'correlation' refers to the statistical measure that describes the extent to which two variables change together. For instance, even if two users rate more items commonly, their correlation might not be as high as compared to another pair because correlation focuses on how the ratings move together.

Let's illustrate with a third user:

JavaScript

// Additional user ratings
const user1RatingsAlt = math.matrix([2, 2, 3, 4, 4]);
const user2RatingsAlt = math.matrix([2, 5, 3, 1, 4]);
const user3RatingsAlt = math.matrix([1, 1, 2, 4, 3]);

// Calculate Pearson correlations
const pearsonSimilarity12 = pearsonCorrelation(user1RatingsAlt, user2RatingsAlt);
const pearsonSimilarity13 = pearsonCorrelation(user1RatingsAlt, user3RatingsAlt);

console.log(`Pearson Correlation between User 1 and User 2: ${pearsonSimilarity12.toFixed(2)}`);  // -0.32
console.log(`Pearson Correlation between User 1 and User 3: ${pearsonSimilarity13.toFixed(2)}`);  // 0.96

// Additional user ratings
const user1RatingsAlt = math.matrix([2, 2, 3, 4, 4]);
const user2RatingsAlt = math.matrix([2, 5, 3, 1, 4]);
const user3RatingsAlt = math.matrix([1, 1, 2, 4, 3]);

// Calculate Pearson correlations
const pearsonSimilarity12 = pearsonCorrelation(user1RatingsAlt, user2RatingsAlt);
const pearsonSimilarity13 = pearsonCorrelation(user1RatingsAlt, user3RatingsAlt);

console.log(`Pearson Correlation between User 1 and User 2: ${pearsonSimilarity12.toFixed(2)}`);  // -0.32
console.log(`Pearson Correlation between User 1 and User 3: ${pearsonSimilarity13.toFixed(2)}`);  // 0.96

Despite having more common ratings with User 2, User 1’s ratings' movement is more aligned with User 3’s. This illustrates how correlation is about the trend in ratings rather than the sheer count of mutual ratings.

This fact is not an advantage or disadvantage of this similarity measure, but it is a fact that you should be aware when using it. There are other metrics that do not have this feature, like cosine similarity.

Introduction to Similarity Measures in Recommendation Systems

Recap of Essential Setup Steps

Understanding Pearson Correlation

Join the 1M+ learners on CodeSignal

Step-by-step Implementation

Aligning Ratings on Co-Rated Items

Example Application and Interpretation

Understanding Correlation with Additional Example

Overview and Preparation for Practice