Implementing Implicit Alternating Least Squares (IALS)

Introduction to IALS

Welcome to the next lesson of this course, where we delve into Implementing Implicit Alternating Least Squares (IALS). Throughout this course, we've progressively constructed a foundation for understanding recommendation systems, moving from explicit rating matrices to utilizing implicit feedback. IALS, our focus for this lesson, is a sophisticated method that leverages implicit data, such as user clicks or views, rather than explicit ratings, to refine recommendations. Let’s explore how this powerful algorithm can elevate your recommendation capabilities by incorporating implicit user preferences.

Recap: Preference and Confidence Matrices

Before we dive deeper into IALS, let's quickly revisit the concepts of preference and confidence matrices. These matrices were initialized from the rating matrix, as you may recall from earlier lessons. The preference matrix indicates whether a user has interacted with an item, while the confidence matrix reflects the certainty of these interactions. Here's a succinct code snippet demonstrating the setup: Python import numpy as np # User-item interaction ratings matrix watch_times_matrix = np.array([...]) # Create preference and confidence matrices preference_matrix = (watch_times_matrix > 0).astype(np.float32) alpha_conf = 40 confidence_matrix = 1 + alpha_conf * watch_times_matrix In this context, each element in the preference_matrix becomes 1 if there is an interaction (non-zero rating), and the confidence_matrix is adjusted to reflect our certainty about these interactions, multiplied by a confidence level alpha_conf .

Optimization Problem

\min_{U,V} \sum_{u,i} c_{ui} (p_{ui} - U_u \cdot V_i^T)^2 + \lambda (\| U_u \|^2 + \| V_i \|^2)

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Python

def update_user_features(user_feat, item_feat, confidence, preference, num_usrs, num_feats, reg_param):
    item_features_T = item_feat.T
    for user in range(num_usrs):
        conf_u = confidence[user]
        conf_u_mat = np.diag(conf_u)
        A = item_features_T @ conf_u_mat @ item_feat + lambda_identity
        b = item_features_T @ conf_u_mat @ preference[user]
        user_feat[user] = np.linalg.solve(A, b)

def update_user_features(user_feat, item_feat, confidence, preference, num_usrs, num_feats, reg_param):
    item_features_T = item_feat.T
    for user in range(num_usrs):
        conf_u = confidence[user]
        conf_u_mat = np.diag(conf_u)
        A = item_features_T @ conf_u_mat @ item_feat + lambda_identity
        b = item_features_T @ conf_u_mat @ preference[user]
        user_feat[user] = np.linalg.solve(A, b)

Python

def update_item_features(user_feat, item_feat, confidence, preference, num_items, num_feats, reg_param):
    user_features_T = user_feat.T
    for item in range(num_items):
        conf_i = confidence[:, item]
        conf_i_mat = np.diag(conf_i)
        A = user_features_T @ conf_i_mat @ user_feat + lambda_identity
        b = user_features_T @ conf_i_mat @ preference[:, item]
        item_feat[item] = np.linalg.solve(A, b)

def update_item_features(user_feat, item_feat, confidence, preference, num_items, num_feats, reg_param):
    user_features_T = user_feat.T
    for item in range(num_items):
        conf_i = confidence[:, item]
        conf_i_mat = np.diag(conf_i)
        A = user_features_T @ conf_i_mat @ user_feat + lambda_identity
        b = user_features_T @ conf_i_mat @ preference[:, item]
        item_feat[item] = np.linalg.solve(A, b)

Python

import numpy as np

num_users, num_items = watch_times_matrix.shape
num_factors = 200  # Reduced from 200 to improve speed while maintaining accuracy
lambda_reg = 40
alpha_conf = 40
num_iterations = 15

# Initialize user and item feature matrices with better initial values
user_features = np.random.normal(0, 0.01, (num_users, num_factors))
item_features = np.random.normal(0, 0.01, (num_items, num_factors))

# Create preference and confidence matrices
preference_matrix = (watch_times_matrix > 0).astype(np.float32)
confidence_matrix = 1 + alpha_conf * watch_times_matrix

# Pre-compute identity matrix
lambda_identity = lambda_reg * np.eye(num_factors, dtype=np.float32)

def train_ials():
    global user_features, item_features
    
    for _ in range(num_iterations):
        update_user_features(user_features, item_features, confidence_matrix, preference_matrix, num_users, num_factors, lambda_reg)
        update_item_features(user_features, item_features, confidence_matrix, preference_matrix, num_items, num_factors, lambda_reg)

    return user_features @ item_features.T

prediction_matrix = train_ials()
print('Final Predicted Ratings Matrix:')
print(prediction_matrix)

import numpy as np

num_users, num_items = watch_times_matrix.shape
num_factors = 200  # Reduced from 200 to improve speed while maintaining accuracy
lambda_reg = 40
alpha_conf = 40
num_iterations = 15

# Initialize user and item feature matrices with better initial values
user_features = np.random.normal(0, 0.01, (num_users, num_factors))
item_features = np.random.normal(0, 0.01, (num_items, num_factors))

# Create preference and confidence matrices
preference_matrix = (watch_times_matrix > 0).astype(np.float32)
confidence_matrix = 1 + alpha_conf * watch_times_matrix

# Pre-compute identity matrix
lambda_identity = lambda_reg * np.eye(num_factors, dtype=np.float32)

def train_ials():
    global user_features, item_features
    
    for _ in range(num_iterations):
        update_user_features(user_features, item_features, confidence_matrix, preference_matrix, num_users, num_factors, lambda_reg)
        update_item_features(user_features, item_features, confidence_matrix, preference_matrix, num_items, num_factors, lambda_reg)

    return user_features @ item_features.T

prediction_matrix = train_ials()
print('Final Predicted Ratings Matrix:')
print(prediction_matrix)

Python

def update_user_features(user_feat, item_feat, confidence, preference, num_usrs, num_feats, reg_param):
    item_features_T = item_feat.T
    for user in range(num_usrs):
        conf_u = confidence[user]
        conf_u_mat = np.diag(conf_u)
        A = item_features_T @ conf_u_mat @ item_feat + lambda_identity
        b = item_features_T @ conf_u_mat @ preference[user]
        user_feat[user] = np.linalg.solve(A, b)

Python

def update_item_features(user_feat, item_feat, confidence, preference, num_items, num_feats, reg_param):
    user_features_T = user_feat.T
    for item in range(num_items):
        conf_i = confidence[:, item]
        conf_i_mat = np.diag(conf_i)
        A = user_features_T @ conf_i_mat @ user_feat + lambda_identity
        b = user_features_T @ conf_i_mat @ preference[:, item]
        item_feat[item] = np.linalg.solve(A, b)

Python

import numpy as np

num_users, num_items = watch_times_matrix.shape
num_factors = 200  # Reduced from 200 to improve speed while maintaining accuracy
lambda_reg = 40
alpha_conf = 40
num_iterations = 15

# Initialize user and item feature matrices with better initial values
user_features = np.random.normal(0, 0.01, (num_users, num_factors))
item_features = np.random.normal(0, 0.01, (num_items, num_factors))

# Create preference and confidence matrices
preference_matrix = (watch_times_matrix > 0).astype(np.float32)
confidence_matrix = 1 + alpha_conf * watch_times_matrix

# Pre-compute identity matrix
lambda_identity = lambda_reg * np.eye(num_factors, dtype=np.float32)

def train_ials():
    global user_features, item_features
    
    for _ in range(num_iterations):
        update_user_features(user_features, item_features, confidence_matrix, preference_matrix, num_users, num_factors, lambda_reg)
        update_item_features(user_features, item_features, confidence_matrix, preference_matrix, num_items, num_factors, lambda_reg)

    return user_features @ item_features.T

prediction_matrix = train_ials()
print('Final Predicted Ratings Matrix:')
print(prediction_matrix)

Introduction to IALS

Recap: Preference and Confidence Matrices

Optimization Problem

Join the 1M+ learners on CodeSignal

Solving with Implicit Alternating Least Squares

Update User Features Function

Detailed Explanation:

Update Item Features Function

Walking Through the Complete IALS Code

Evaluating IALS

Optimisation

Summary and Preparing for Practice