Welcome to this lesson on factorization machines, an important model in the realm of recommendation systems. Factorization machines (FM) excel at capturing interactions between variables, making them a powerful tool for both regression and classification tasks. For instance, they can predict a rating (regression) or calculate the likelihood of a recommendation (classification).

Before we dive into implementing a factorization machine, let's briefly revisit the dataset preparation process from the previous lesson.

Previously, you learned how to load data from JSON files and represent it as arrays of objects in JavaScript. You also created a user-item interaction matrix using dummy variables (one-hot encoding) and enriched the dataset with auxiliary features such as user preferences and genre similarity. These steps are crucial for building a dataset that can be used for accurate predictions in a recommendation system.

For this lesson, assume your data is structured as an array of objects, where each object represents a user-item interaction with features like:

{
  user1: 0, user2: 1, user3: 0, // one-hot user
  item1: 0, item2: 0, item3: 1, // one-hot item
  uf1: 0.7, uf2: 0.2,           // user features
  if1: 0.5, if2: 0.1,           // item features
  rating: 4.0                   // target value
}

To train the model, you will need to convert this array of objects into two arrays:

• X: an array of arrays, where each sub-array contains the feature values for one interaction (excluding the target).
• y: an array of target values (e.g., ratings).

Let's start by defining the class and initializing the required data in JavaScript. We'll use mathjs for vector operations and ml-matrix for matrix handling, as in the practice section.

import { Matrix } from 'ml-matrix';
import * as math from 'mathjs';

class SimpleFactorizationMachine {
    constructor(nFactors, nFeatures, learningRate = 0.01, epochs = 100, reg = 0.01) {
        this.nFactors = nFactors;
        this.learningRate = learningRate;
        this.epochs = epochs;
        this.reg = reg;

        this.w0 = 0;
        this.W = Array(nFeatures).fill(0);
        this.V = Array.from({ length: nFeatures }, () =>
            Array.from({ length: nFactors }, () => math.random(-0.1, 0.1))
        );
    }
}

• nFactors: Number of components in each latent vector (latent space dimensionality).
• nFeatures: Total number of features in the dataset.
• W: Linear coefficients for each feature.
• V: Matrix of latent vectors for each feature, initialized randomly.

Now, let's implement the fit method in JavaScript using ml-matrix and mathjs for correct and efficient vector/matrix operations.

SimpleFactorizationMachine.prototype.fit = function(X, y) {
    const m = X.rows;
    const n = X.columns;
    for (let epoch = 0; epoch < this.epochs; epoch++) {
        for (let i = 0; i < m; i++) {
            let x = X.getRow(i);
            let linearTerms = this.w0 + math.dot(x, this.W);

            let interactionTerm = 0;
            for (let f = 0; f < this.nFactors; f++) {
                let v_f = this.V.map(row => row[f]);
                let dot_x_vf = math.dot(x, v_f);
                let dot_x2_vf2 = math.dot(x.map(xi => xi ** 2), v_f.map(vfi => vfi ** 2));
                interactionTerm += (dot_x_vf ** 2 - dot_x2_vf2) / 2;
            }

            let prediction = linearTerms + interactionTerm;
            let err = prediction - y[i];

            this.w0 -= this.learningRate * err;

            for (let j = 0; j < n; j++) {
                this.W[j] -= this.learningRate * (err * X.get(i, j) + this.reg * this.W[j]);
            }

            for (let f = 0; f < this.nFactors; f++) {
                let v_f = this.V.map(row => row[f]);
                for (let j = 0; j < n; j++) {
                    let grad = err * (x[j] * (math.dot(x, v_f) - v_f[j] * x[j])) + this.reg * this.V[j][f];
                    this.V[j][f] -= this.learningRate * grad;
                }
            }
        }
    }
};

• Linear Terms: Calculated as the sum of the global bias and the dot product of the feature vector and linear coefficients.
• Interaction Terms: For each latent factor, sum the dot products and squared terms to capture feature interactions.
• Parameter Updates: Update the global bias, linear coefficients, and interaction factors using gradient descent and regularization.

Now, let's implement the predict method in JavaScript, again using ml-matrix and mathjs.

SimpleFactorizationMachine.prototype.predict = function(X) {
    const m = X.rows;
    const n = X.columns;
    let yPred = Array(m).fill(0);
    for (let i = 0; i < m; i++) {
        let x = X.getRow(i);
        let linearTerms = this.w0 + math.dot(x, this.W);
        let interactionTerm = 0;
        for (let f = 0; f < this.nFactors; f++) {
            let v_f = this.V.map(row => row[f]);
            let dot_x_vf = math.dot(x, v_f);
            let dot_x2_vf2 = math.dot(x.map(xi => xi ** 2), v_f.map(vfi => vfi ** 2));
            interactionTerm += (dot_x_vf ** 2 - dot_x2_vf2) / 2;
        }
        yPred[i] = linearTerms + interactionTerm;
    }
    return yPred;
};

• For each data instance, compute the linear and interaction terms, then sum them to get the prediction.

In this lesson, you implemented and evaluated a factorization machine model for recommendation systems in JavaScript. You learned how to initialize model parameters, train the model using gradient descent, make predictions, and evaluate performance. This concludes our exploration of factorization machines and marks the end of this course module.

Congratulations on completing the course! The skills you've acquired here form a strong foundation for building and understanding recommendation systems. Continue exploring other models and refining your expertise in this dynamic field. Well done!