Improved Prediction Using Adjusted Weighted Average

Welcome to the final lesson of this course on recommendation systems, where we will explore the concept of adjusted weighted averages. Previously, we've used raw ratings to predict user preferences. However, this approach can introduce bias, as it doesn't account for individual users' rating tendencies. In this lesson, you'll learn how switching to using the difference between a rating and a user's average rating can improve prediction accuracy by minimizing these biases.

Let's briefly revisit the code setup that we've built upon throughout this course. You should already be familiar with reading a user-item rating matrix from a text file and setting the stage for using this data in predictions. Here's a quick code reminder:

import numpy as np

# Read user-item rating matrix from a text file
def read_users_items_matrix(file_path):
    users_items_matrix = {}
    with open(file_path, 'r') as file:
        for line in file:
            user, item, rating = line.strip().split(',')
            if user not in users_items_matrix:
                users_items_matrix[user] = {}
            users_items_matrix[user][item] = int(rating)
    return users_items_matrix

# Define the path to the text file
file_path = 'user_items_matrix.txt'

# Read the user-item matrix
users_items_matrix = read_users_items_matrix(file_path)

This code reads the user-item matrix from a file, setting up our essential data structure for further manipulations. Remember, understanding this setup is crucial as we now proceed to modify our prediction approach.

When we use raw ratings in recommendation systems, we might introduce bias because different users have different rating tendencies. Here's what that means:

• Consistently High Raters: Some users might generally give high ratings to most items, regardless of their true preference. For example, a user might rate most movies 4 or 5 stars.
• Consistently Low Raters: Conversely, some users might rate items lower on average, even if they like them. They might give most movies 2 or 3 stars.

These tendencies can skew predictions because the system might interpret a high rating as a strong preference, even if it's just the user's habit. To reduce this bias and improve the accuracy of our recommendation system, we adjust the ratings by subtracting the average rating of each user.

By using the rating differences rather than raw averages, we can better identify genuine preferences:

• This adjustment ensures that predictions are based more on relative preferences rather than absolute ratings.
• It helps to normalize user ratings, making comparisons between users more equitable.