Locally Linear Embedding in R

Welcome to the fascinating world of Locally Linear Embedding (LLE), a vital tool in our dimensionality reduction toolbox. Unlike linear techniques like Principal Component Analysis (PCA), LLE shines at preserving local neighborhood structure in high-dimensional data.

In this lesson, we’ll unpack the LLE algorithm, discuss when to use it, and contrast it with PCA. We’ll implement it in R, using ggplot2 for visualization and a small helper we’ll write to run LLE.

LLE preserves relationships within local neighborhoods while reducing dimensionality, capturing twists and turns in non-linear manifolds (e.g., images, pose, genomics). Like reading a street map vs. a bird’s-eye projection: PCA may distort local distances, whereas LLE maintains them.

We’ll use the Swiss Roll—a classic 3D manifold that’s hard for linear methods but perfect for LLE.

# ---- Swiss Roll & LLE in R (from scratch) ----
# install.packages(c("ggplot2"))  # run once if needed

library(ggplot2)

# Generate Swiss Roll data (like sklearn.datasets.make_swiss_roll)
generate_swiss_roll <- function(n_samples = 1500, noise = 0.05, seed = 42) {
  set.seed(seed)
  t <- 1.5 * pi * (1 + 2 * runif(n_samples))
  h <- 21 * runif(n_samples)
  X <- cbind(
    x = t * cos(t) + rnorm(n_samples, sd = noise),
    y = h           + rnorm(n_samples, sd = noise),
    z = t * sin(t) + rnorm(n_samples, sd = noise)
  )
  list(X = X, color = t)
}

n_samples <- 1500
noise <- 0.05
swiss <- generate_swiss_roll(n_samples, noise, seed = 42)
X <- swiss$X
color <- swiss$color