Preparing Data for Recommendations

Introduction: What Are Factorization Machines?

Welcome to the lesson on preparing datasets for factorization machines . In this lesson, you will learn how to create a detailed dataset to be used in recommendation systems using factorization machines. Factorization machines are advanced models that capture complex interactions between different data features, making them powerful tools for making accurate recommendations.

Bridging Content-Based Methods and Factorization Machines

Previously, you learned how to build content-based recommendation systems using heuristics—such as matching user preferences to item features (e.g., genres, popularity, or listening averages). While these methods are intuitive and useful, they often struggle to model the complex, sparse, and high-dimensional nature of real-world user-item interactions. To address these limitations, we move to supervised models like factorization machines. Factorization machines are designed to handle sparse data and can automatically learn pairwise interactions between any features (not just user and item IDs, but also auxiliary features). This allows them to capture subtle patterns and improve recommendation accuracy, especially when explicit interactions are limited. Why focus on a structured dataset? A well-prepared dataset allows a factorization machine to learn meaningful relationships from the data, leading to better recommendation outcomes. This lesson will guide you through organizing your data in a format suitable for factorization machines.

Recap: Initial Setup and Data Overview

Before diving into dataset preparation, let's briefly review how to read and understand our data files. You will work with three JSON files: tracks.json, users.json, and interactions.json. Each file contains structured information about tracks, users, and their interactions. Here is an example of what the interactions.json file might look like: JSON[ { "user_id": 1, "track_id": 1, "rating": 3 }, { "user_id": 1, "track_id": 2, "rating": 4 } // ... more data ] For each pair of a user and a track that this user interacted with, the file keeps track of the rating that the user gave to this track. Note: In earlier lessons, you may have seen examples where IDs (such as track_id or user_id) were represented as strings (e.g., "track_id": "001"). Starting in this unit, and for the remainder of the course, we use integer IDs (e.g., "id": 1). This change is intentional: using integer IDs makes it easier to create dummy variables, build matrices, and perform efficient indexing in Go. Integer IDs are also more common in real-world machine learning datasets, especially when preparing data for models like factorization machines. To load these files in Go, you can use the os and encoding/json packages. Here is a consolidated code snippet to read the JSON files and unmarshal them into Go structs: Gopackage main import ( "encoding/json" "fmt" "log" "os" ) type Track struct { ID int `json:"id"` Title string `json:"title"` Likes int `json:"likes"` Genre string `json:"genre"` } type User struct { ID int `json:"id"` Name string `json:"name"` TimeListeningAvg int `json:"time_listening_avg"` GenrePreferences map[string]float64 `json:"genre_preferences"` } type Interaction struct { UserID int `json:"user_id"` TrackID int `json:"track_id"` Rating int `json:"rating"` } func main() { // Read tracks.json tracksBytes, err := os.ReadFile("tracks.json") if err != nil { log.Fatal(err) } var tracks []Track if err := json.Unmarshal(tracksBytes, &tracks); err != nil { log.Fatal(err) } // Read users.json usersBytes, err := os.ReadFile("users.json") if err != nil { log.Fatal(err) } var users []User if err := json.Unmarshal(usersBytes, &users); err != nil { log.Fatal(err) } // Read interactions.json interactionsBytes, err := os.ReadFile("interactions.json") if err != nil { log.Fatal(err) } var interactions []Interaction if err := json.Unmarshal(interactionsBytes, &interactions); err != nil { log.Fatal(err) } fmt.Println(tracks) fmt.Println(users) fmt.Println(interactions) } This code reads the JSON files and loads their contents into Go slices of structs. This is a common first step before further processing in Go.

Creating the User-Item Interaction Matrix

The user-item interaction matrix is a fundamental component of many recommendation systems. It's a simplified way to understand which user interacts with which item and how. Imagine you have three users and three tracks. Each interaction can be represented using dummy variables that indicate whether a user interacted with a track and what their rating was. For example: user1 user2 user3 track1 track2 track3 rating 10010030100104 In this table, 1 and 0 indicate the presence or absence of interaction between users and tracks. The rating column shows the rating a user gave to a track. This representation allows us to define user-item pairs. For example, the first row is for the user1 and track1 pair, and it tells us that user1's rating for track1 is 3 .

user1	user2	user3	track1	track2	track3	rating
1	0	0	1	0	0	3
0	1	0	0	1	0	4

Adding Auxiliary Features: Overview

To make our dataset more informative, we incorporate auxiliary features such as user preferences and item statistics. These enrich the data, providing a better foundation for the system to learn from. Let's consider an example: user1 user2 user3 track1 track2 track3 track_likes user_listening_avg genre_similarity rating 100100100180 0.89 3 010010250220 0.76 4 In the table, besides the user and track dummy variables and the rating column, we have additional auxiliary features: track_likes : Indicates the number of likes a track has received, providing a measure of popularity. user_listening_avg : Reflects the average listening duration for a user, which can indicate user engagement. genre_similarity : Measures the similarity between a user's genre preferences and a track's genre using cosine similarity. These features enrich the dataset, allowing factorization machines to capture deeper insights and nuances within the data.

user1	user2	user3	track1	track2	track3	track_likes	user_listening_avg	genre_similarity	rating
1	0	0	1	0	0	100	180	0.89	3
0	1	0	0	1	0	250	220	0.76	4

Adding Auxiliary Features: Computing Similarity

To compute the similarity between a user's genre preferences and a track's genre, we first need to encode genres and then calculate the cosine similarity between the user's genre preference vector and the track's genre vector. In Go, we can represent genre encodings as a map from genre name to a slice of float64 values (one-hot encoding). We can then write a function to compute cosine similarity between two slices. Goimport ( "math" ) // Map genres to one-hot encoded vectors var genreEncodings = map[string][]float64{ "Jazz": {1, 0, 0}, "Pop": {0, 1, 0}, "Rock": {0, 0, 1}, } // CosineSimilarity computes the cosine similarity between two vectors func CosineSimilarity(a, b []float64) float64 { var dotProduct, normA, normB float64 for i := 0; i < len(a); i++ { dotProduct += a[i] * b[i] normA += a[i] * a[i] normB += b[i] * b[i] } if normA == 0 || normB == 0 { return 0 } return dotProduct / (math.Sqrt(normA) * math.Sqrt(normB)) } To calculate the genre similarity for a user and a track: Go// Example user genre preferences userGenreArray := []float64{0.5, 0.8, 0.3} // [Jazz, Pop, Rock] // Example track genre trackGenreArray := genreEncodings["Jazz"] genreSimilarity := CosineSimilarity(userGenreArray, trackGenreArray) fmt.Printf("Genre similarity: %.2f\n", genreSimilarity) This value reflects how closely user preferences match a track's genre, which can significantly influence recommendations.

Constructing the Data Matrix: Creating Dummy Variables

In this part, we create dummy variables for users and tracks. Dummy variables are binary indicators that represent whether a particular user or track is involved in the interaction. In Go, you can use for loops and slices to create these dummy variables: Go // Suppose we have 3 users and 3 tracks numUsers := 3 numTracks := 3 userID := 1 // For example, user with ID 1 trackID := 2 // For example, track with ID 2 // Create user dummy variables userDummies := make([]int, numUsers) for i := 0; i < numUsers; i++ {if i+1 == userID {userDummies[i] = 1} else {userDummies[i] = 0}} // Create track dummy variables trackDummies := make([]int, numTracks) for i := 0; i < numTracks; i++ {if i+1 == trackID {trackDummies[i] = 1} else {trackDummies[i] = 0}} userDummies is a slice where the position corresponding to the current user is set to 1, and all others are 0 . trackDummies is a slice where the position corresponding to the current track is set to 1, and all others are 0 .

Constructing the Data Matrix: Extracting Features

Next, we extract specific features from both user and track data using their respective IDs. In Go, you can use for loops to find the matching user and track structs. Go// Find user by ID var user User for _, u := range users { if u.ID == userID { user = u break } } // Find track by ID var track Track for _, t := range tracks { if t.ID == trackID { track = t break } } trackLikes := track.Likes userListeningAvg := user.TimeListeningAvg trackLikes is extracted to indicate the number of likes a track has. userListeningAvg is used to measure the average listening duration for a user.

Constructing the Data Matrix: Calculating Genre Similarity and Combining Features

This section calculates the genre similarity and consolidates all features into a single row. Go// Prepare user genre preferences as a slice in the order [Jazz, Pop, Rock] userGenreArray := []float64{ user.GenrePreferences["Jazz"], user.GenrePreferences["Pop"], user.GenrePreferences["Rock"], } // Get track genre one-hot encoding trackGenreArray := genreEncodings[track.Genre] // Calculate genre similarity genreSimilarity := CosineSimilarity(userGenreArray, trackGenreArray) // Combine all features into a single row row := []float64{} for _, v := range userDummies { row = append(row, float64(v)) } for _, v := range trackDummies { row = append(row, float64(v)) } row = append(row, float64(trackLikes)) row = append(row, float64(userListeningAvg)) row = append(row, genreSimilarity) row = append(row, float64(rating)) // rating from the interaction All dummy variables and features are consolidated into a single row, which can be appended to a data matrix.

Constructing the Data Matrix: Complete Code Snippet

The final step is to gather all these elements into a structured data matrix. In Go, you can use a slice of slices to represent the matrix, or define a struct for each row if you want to keep column names. Here is a complete example that builds the data matrix as a slice of slices: Gopackage main import ( "encoding/json" "fmt" "log" "math" "os" ) type Track struct { ID int `json:"id"` Title string `json:"title"` Likes int `json:"likes"` Genre string `json:"genre"` } type User struct { ID int `json:"id"` Name string `json:"name"` TimeListeningAvg int `json:"time_listening_avg"` GenrePreferences map[string]float64 `json:"genre_preferences"` } type Interaction struct { UserID int `json:"user_id"` TrackID int `json:"track_id"` Rating int `json:"rating"` } // CosineSimilarity computes the cosine similarity between two vectors func CosineSimilarity(a, b []float64) float64 { var dotProduct, normA, normB float64 for i := 0; i < len(a); i++ { dotProduct += a[i] * b[i] normA += a[i] * a[i] normB += b[i] * b[i] } if normA == 0 || normB == 0 { return 0 } return dotProduct / (math.Sqrt(normA) * math.Sqrt(normB)) } func main() { // Load data as shown in the previous section // ... (loading tracks, users, interactions) // For brevity, assume tracks, users, and interactions are already loaded // Example genre encodings genreEncodings := map[string][]float64{ "Jazz": {1, 0, 0}, "Pop": {0, 1, 0}, "Rock": {0, 0, 1}, } numUsers := len(users) numTracks := len(tracks) // Build the data matrix var data [][]float64 for _, interaction := range interactions { userID := interaction.UserID trackID := interaction.TrackID rating := interaction.Rating // Create user dummy variables userDummies := make([]int, numUsers) for i := 0; i < numUsers; i++ { if users[i].ID == userID { userDummies[i] = 1 } else { userDummies[i] = 0 } } // Create track dummy variables trackDummies := make([]int, numTracks) for i := 0; i < numTracks; i++ { if tracks[i].ID == trackID { trackDummies[i] = 1 } else { trackDummies[i] = 0 } } // Find user and track var user User for _, u := range users { if u.ID == userID { user = u break } } var track Track for _, t := range tracks { if t.ID == trackID { track = t break } } // Extract features trackLikes := track.Likes userListeningAvg := user.TimeListeningAvg // Prepare user genre preferences as a slice in the order [Jazz, Pop, Rock] userGenreArray := []float64{ user.GenrePreferences["Jazz"], user.GenrePreferences["Pop"], user.GenrePreferences["Rock"], } // Get track genre one-hot encoding trackGenreArray := genreEncodings[track.Genre] // Calculate genre similarity genreSimilarity := CosineSimilarity(userGenreArray, trackGenreArray) // Combine all features into a single row row := []float64{} for _, v := range userDummies { row = append(row, float64(v)) } for _, v := range trackDummies { row = append(row, float64(v)) } row = append(row, float64(trackLikes)) row = append(row, float64(userListeningAvg)) row = append(row, genreSimilarity) row = append(row, float64(rating)) data = append(data, row) } // Print the data matrix // Define column names for reference columns := []string{ "user1", "user2", "user3", // adjust for your number of users "track1", "track2", "track3", // adjust for your number of tracks "track_likes", "user_listening_avg", "genre_similarity", "rating", } fmt.Println(columns) for _, row := range data { fmt.Println(row) } } This code builds a data matrix where each row contains all the dummy variables and features for a user-track interaction. You can use this matrix as input for factorization machines or other recommendation algorithms.

Summary and Preparation for Practice

Congratulations! You have learned how to prepare a dataset for factorization machines by creating a comprehensive data matrix. This matrix serves as the foundation for designing an effective recommendation system. In this lesson, you revisited loading JSON files, learned to represent user-item interactions with dummy variables, and incorporated auxiliary features to enrich the data. Understanding these components is crucial for designing more accurate and personalized recommendation algorithms. Now, it's time for you to apply these skills in the upcoming practice exercises. Your journey into creating sophisticated recommendation systems begins with mastering these foundations. Good luck, and remember to reference this lesson as needed!

Previous Lesson Next Lesson: Factorization Machines in Go

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal