Welcome to the first unit of our course, "Estimate Geometry Using Homographies and RANSAC with Python"! Over the next four units, we are going to learn how to combine overlapping photographs to create a single, wide panorama.

To build a panorama, we need to mathematically align overlapping images. Imagine taking a map, tearing it into two pieces, and then trying to tape them back together. To align them correctly, you look for matching landmarks on both pieces.

In computer vision, the mathematical rule that aligns these two pieces is called a homography. A homography is a transformation matrix that maps points in one image to their corresponding points in another. By calculating this matrix, we can warp and shift our photos so they overlap perfectly.

It is important to note that a homography is the correct model in two specific scenarios: when the scene is roughly planar (like a flat wall) or when the camera undergoes pure rotation around its center (like panned shots on a tripod). If you move the camera's position in a scene with objects at many different depths, you encounter parallax, which can prevent a single homography from perfectly aligning the entire image. In this lesson, we will focus on exactly how to create this homography matrix using Python.

Before we can calculate our homography math, we need to find overlapping points. Imagine our code has already used some helper functions (which we will call detect_and_compute and match_descriptors) to look at two photos, identify interesting spots like corners or edges, and pair them up.

However, we face a big problem: raw matches are often imperfect. The computer might mistakenly match a cloud in the left image to a completely different cloud in the right image. These bad matches are called "outliers." We cannot just trust all of our matches to build our map. We need a smart way to calculate our homography while completely ignoring the bad data.

To figure out our alignment, we first need to extract the exact pixel (x, y) coordinates of our matched points. The match objects provide an index, which we use to look up the actual (x, y) coordinate.

Let's begin writing a function called matched_points. First, we pull out the coordinates using Python list comprehensions.

def matched_points(kp1, kp2, matches):
    src_points = [kp1[m.queryIdx].pt for m in matches]
    dst_points = [kp2[m.trainIdx].pt for m in matches]

In this code:

• kp1 and kp2 are lists of "keypoints" (the interesting spots) for the left and right images.
• m.queryIdx gives us the index of the point in the first image (the source).
• m.trainIdx gives us the index of the matched point in the second image (the target).
• .pt grabs the actual (x, y) pixel coordinates.

Next, we need to format these lists for OpenCV. OpenCV requires these points to be wrapped in a specific NumPy format.

import numpy as np

def matched_points(kp1, kp2, matches):
    src = np.float32([kp1[m.queryIdx].pt for m in matches]).reshape(-1, 1, 2)
    dst = np.float32([kp2[m.trainIdx].pt for m in matches]).reshape(-1, 1, 2)
    return src, dst

Here, we wrap our lists in np.float32 because OpenCV requires 32-bit decimal numbers. We then use .reshape(-1, 1, 2) to structure the data into a 3D array format. This tells OpenCV that we have a list of single points, and each point has 2 values (an X and a Y).

As mentioned, raw matches often contain "outliers" — bad pairs that don't actually correspond to the same point in space. If we tried to calculate our homography using every single match, these outliers would pull the alignment off-center, resulting in a blurry or distorted panorama.

To solve this, we use RANSAC, which stands for RANdom Sample Consensus. It is a robust estimation algorithm that works in an iterative way:

1. Select: It randomly picks the minimum number of points required to calculate a homography (which is 4 pairs).
2. Estimate: It calculates a candidate homography matrix using only those 4 points.
3. Test: It applies this matrix to all the other matches and checks how many of them actually align correctly within a certain error margin (the threshold).
4. Consensus: The matches that align are called inliers. The algorithm repeats this process many times, keeping the homography that produces the highest number of inliers.

By using the "consensus" of the majority of points, RANSAC effectively ignores the "noise" caused by clouds, moving cars, or repetitive patterns that might have fooled our initial matching step.

Now that we have our formatted coordinates, we can calculate the homography. Let's start building a function called estimate_homography.

To calculate a homography, geometry rules dictate that we need at least four pairs of points. If we have fewer than four, it is impossible to perform the math.

def estimate_homography(kp1, kp2, matches, ransac_threshold=5.0):
    if len(matches) < 4:
        raise ValueError("At least four matches are required")

Assuming we have enough matches, we can now use OpenCV to find the homography using the RANSAC method we just discussed.

import cv2
import numpy as np

def estimate_homography(kp1, kp2, matches, ransac_threshold=5.0):
    if len(matches) < 4:
        raise ValueError("At least four matches are required")

    src, dst = matched_points(kp1, kp2, matches)
    homography, mask = cv2.findHomography(src, dst, cv2.RANSAC, ransac_threshold)

We call cv2.findHomography, passing in our src points, dst points, and telling it to use the cv2.RANSAC method.

The ransac_threshold parameter sets our strictness. A value of 5.0 pixels means a transformed point can be up to five pixels away from its matched location and still be counted as a good match (an inlier). If the distance is larger, RANSAC assumes the match is wrong and ignores it.

Finally, we need to handle cases where the estimation fails and then return our final values.

import cv2
import numpy as np

def estimate_homography(kp1, kp2, matches, ransac_threshold=5.0):
    if len(matches) < 4:
        raise ValueError("At least four matches are required")

    src, dst = matched_points(kp1, kp2, matches)
    homography, mask = cv2.findHomography(src, dst, cv2.RANSAC, ransac_threshold)

    if homography is None or mask is None:
        raise ValueError("Homography estimation failed")

    return homography, mask.ravel().astype(bool)

If OpenCV cannot find a valid transformation, it returns None. We check for this to prevent errors. Finally, we return the homography matrix and the mask.

A mask in computer vision acts like a filter. In this case, cv2.findHomography returns an array of the same length as our input matches. Each element is either a 1 (indicating the match is an inlier that fits the transformation) or a 0 (indicating it is an outlier). We flatten this into a simple 1D boolean array where True marks the good matches and False marks the bad ones.

Let's see how this all connects in an application. Imagine we have a script that reads two images and extracts their features using helper functions.

First, we load our features and find our matches.

# Assume left_image and right_image are already loaded
kp1, des1 = detect_and_compute(left_image)
kp2, des2 = detect_and_compute(right_image)

matches = match_descriptors(des1, des2)

Now, we simply pass these values into our new estimate_homography function and print out our findings.

# Assume left_image and right_image are already loaded
kp1, des1 = detect_and_compute(left_image)
kp2, des2 = detect_and_compute(right_image)

matches = match_descriptors(des1, des2)

homography, inliers = estimate_homography(
    kp1,
    kp2,
    matches,
    ransac_threshold=5.0,
)

print("matches:", len(matches))
print("inliers:", int(inliers.sum()))
print("inlier ratio:", float(inliers.mean()))
print("homography:")
print(homography)

When you run a script like this, you will see an output similar to this:

matches: 120
inliers: 95
inlier ratio: 0.7916666666666666
homography:
[[ 1.250e+00 -1.500e-01  3.000e+02]
 [ 5.000e-02  1.100e+00 -5.000e+01]
 [ 1.000e-04  2.000e-04  1.000e+00]]

In this output, we started with 120 raw matches. RANSAC determined that 95 of them were good (inliers), giving us roughly a 79% success rate. The printed 3x3 matrix at the end is the final homography — the mathematical rule we will use later to warp our images into a panorama!

Great job making it through the first lesson of the course! Let's review what we just built.

We learned how to transform raw matches into a geometrical alignment. We formatted point coordinates so OpenCV could read them and explored the geometric conditions (planar scenes or pure rotation) where a homography is applicable. We also learned that we need at least four points to map out a transformation and used RANSAC to filter out the bad matches.

Now, it is time to put this into practice! In the upcoming interactive exercises, you will write and test this exact code yourself. You do not need to worry about installing libraries on your own machine right now; the CodeSignal IDE has NumPy and OpenCV pre-installed and ready to go, so you can focus entirely on writing the code.