Welcome to Unit 4 of our course. Up to this point, you have learned how to preprocess images and extract visual fingerprints by detecting keypoints and computing their descriptors.

Now, we are ready to tackle the first real dependency in our image stitching pipeline: feature matching. Finding keypoints in an isolated image is a great start, but to stitch a panorama, we must find the exact same points across two different views. By matching these descriptors, we create the critical links needed to eventually align and stitch images.

In this lesson, we will build a reliable feature matcher using OpenCV. We will configure our matcher based on descriptor data type and use Lowe's Ratio Test, a powerful technique for filtering out ambiguous matches.

Different algorithms produce different descriptor formats. SIFT produces arrays of floating-point numbers. ORB and AKAZE produce compact binary descriptors, represented as 8-bit unsigned integers.

Because our pipeline supports all three methods, our matcher needs to inspect the descriptor type before choosing a matching strategy.

import numpy as np

def descriptors_are_binary(descriptors):
    return descriptors is not None and descriptors.dtype == np.uint8

If the descriptor dtype is np.uint8, we treat it as binary. Otherwise, we treat it as a floating-point descriptor such as SIFT.

When matching descriptors, the algorithm computes distances between numeric fingerprints. A shorter distance means two descriptors look more similar.

For each descriptor in the first image, we ask for the two nearest candidates in the second image. The best candidate should be clearly better than the runner-up. If the two candidates are too close in quality, the match is ambiguous and should be rejected.

Descriptor from image A
        |
        |-- best candidate in image B      distance = 30
        |
        |-- second-best candidate in B     distance = 80

Ratio check:
best distance < ratio * second-best distance

With ratio = 0.75:
30 < 0.75 * 80
30 < 60  -> keep the match

Now compare that with an ambiguous case:

Descriptor from image A
        |
        |-- best candidate in image B      distance = 55
        |
        |-- second-best candidate in B     distance = 60

With ratio = 0.75:
55 < 0.75 * 60
55 < 45  -> reject the match

The ratio acts like a strictness dial:

• Lower values, such as 0.65, are stricter and produce fewer matches.
• Higher values, such as 0.85, are looser and may keep more false matches.

The core filtering loop is short:

good = []
for pair in raw_matches:
    if len(pair) == 2:
        best, second = pair
        if best.distance < ratio * second.distance:
            good.append(best)

Now, let's build our full match_descriptors function step by step. First, we will check our inputs and use our helper function to see if we have binary descriptors.

import cv2
import numpy as np

def match_descriptors(des1, des2, ratio=0.75):
    if des1 is None or des2 is None:
        return []

    if len(des1) < 2 or len(des2) < 2:
        return []

    binary = descriptors_are_binary(des1) or descriptors_are_binary(des2)

Next, we need to configure OpenCV's Fast Library for Approximate Nearest Neighbors (FLANN) matcher. This is a highly optimized matcher, but it requires specific configuration dictionaries based on our data type.

    if binary:
        # Settings for binary descriptors (ORB, AKAZE)
        index_params = dict(
            algorithm=6,
            table_number=6,
            key_size=12,
            multi_probe_level=1,
        )
        des1 = np.asarray(des1, dtype=np.uint8)
        des2 = np.asarray(des2, dtype=np.uint8)
    else:
        # Settings for floating-point descriptors (SIFT)
        index_params = dict(algorithm=1, trees=5)
        des1 = np.asarray(des1, dtype=np.float32)
        des2 = np.asarray(des2, dtype=np.float32)

    matcher = cv2.FlannBasedMatcher(index_params, dict(checks=50))

Here, we provide the exact algorithm parameters that OpenCV requires to process either binary or floating-point data efficiently. We also ensure our arrays are converted to the correct dtype format.

Now, we can ask the matcher for our top two candidates, apply the ratio test loop we learned earlier, and sort the results.

    raw_matches = matcher.knnMatch(des1, des2, k=2)

    good = []
    for pair in raw_matches:
        if len(pair) == 2:
            best, second = pair
            if best.distance < ratio * second.distance:
                good.append(best)

    # Sort the matches so the best ones (shortest distance) are first
    return sorted(good, key=lambda match: match.distance)

By returning the matches sorted by match.distance, the most confident pairs are always placed at the beginning of our list.

To verify our work, we can use OpenCV's cv2.drawMatches function. This takes our two images, their keypoints, and our list of good matches, and draws lines connecting the corresponding features.

# Assuming left and right images, along with their keypoints (kp) and descriptors (des) are ready
matches = match_descriptors(des1, des2, ratio=0.75)

print("method: sift")
print("ratio: 0.75")
print("left keypoints:", len(kp1))
print("right keypoints:", len(kp2))
print("good matches:", len(matches))

preview = cv2.drawMatches(
    left, kp1,
    right, kp2,
    matches[:60], # Draw only the top 60 to avoid clutter
    None,
    flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS,
)

When you run this on a pair of images, your output will look something like this:

method: sift
ratio: 0.75
left keypoints: 2000
right keypoints: 2000
good matches: 438

The resulting preview image will beautifully display the two photos side by side with colorful lines connecting the exact same locations in both views.

In this lesson, you took a major step forward in building your image stitcher. You learned how to inspect descriptor data types, configure a FLANN matcher for floating-point or binary descriptors, and apply Lowe's Ratio Test to filter unreliable matches.

This matching process provides the essential links we need. In our next and final unit, we will turn matching into a repeatable diagnostic report so we can warn the user before attempting more fragile geometry steps such as homography estimation.

Now, head over to the upcoming practice exercises. You will write the matching logic yourself and experiment with how different ratio values affect the number and quality of good matches.