Welcome back! So far in this course, we have laid a lot of the groundwork. In Units 1 and 2, you learned how to find keypoints in two different images, match them, and use the RANSAC algorithm to calculate an accurate homography matrix.

Now, here in Unit 3, we are going to actually put that homography matrix to use. A homography becomes visible when we use it to warp pixels. To do this, we need to assign roles to our two images:

• Source image: The image being stretched and moved (usually the left image).
• Target image: The image that stays fixed in place (usually the right image).

Think of the source image as a photo printed on a rubber sheet. The homography matrix is the mathematical rule telling us exactly how to pull, stretch, and move that rubber sheet so its landmarks perfectly overlap with a second photo sitting on the table.

Let's learn how to apply this rule using Python and OpenCV.

When we stretch and shift our source image, its pixels will move to new coordinates. A common problem is that some pixels might be pushed into negative coordinates (off the top or left of the screen) or past the right edge. If we do not make our image "canvas" larger to make room, the warped image will be cut off.

To fix this, we first need to determine exactly where the four corners of our source image will be located after the warp. Let's start by defining the original corners of both images.

import cv2
import numpy as np

# Let's assume 'source' and 'target' are already loaded images
h1, w1 = source.shape[:2]
h2, w2 = target.shape[:2]

# Define the 4 corners of the source image [x, y]
source_corners = np.float32(
    [[0, 0], [w1, 0], [w1, h1], [0, h1]]
).reshape(-1, 1, 2)

# Define the 4 corners of the target image [x, y]
target_corners = np.float32(
    [[0, 0], [w2, 0], [w2, h2], [0, h2]]
).reshape(-1, 1, 2)

In OpenCV, functions that process geometric points often require a very specific 3D shape. We use .reshape(-1, 1, 2) to format our corners into the exact shape OpenCV expects.

Next, we will apply our homography rule to the source_corners to see where they move. We use cv2.perspectiveTransform, which applies the mathematical rule to specific coordinate points rather than full images.

# Transform the source corners using the homography matrix
warped_corners = cv2.perspectiveTransform(source_corners, homography)

# Combine the new warped corners with the fixed target corners
all_corners = np.concatenate([warped_corners, target_corners], axis=0)

print(all_corners.shape)

Output:
(8, 1, 2)

Now we have a list of all eight corners. To find our new canvas size, we simply need to find the absolute minimum and maximum x and y values among all these points.

# Find the lowest x and y, and the highest x and y
xmin, ymin = np.floor(all_corners.min(axis=0).ravel()).astype(int)
xmax, ymax = np.ceil(all_corners.max(axis=0).ravel()).astype(int)

print(f"Canvas boundaries: xmin={xmin}, ymin={ymin}, xmax={xmax}, ymax={ymax}")

Output:
Canvas boundaries: xmin=-250, ymin=-50, xmax=1000, ymax=600

Because the source image was shifted left and up to align with the target, our xmin and ymin values are negative!

Image files cannot have negative pixel coordinates. A pixel at x = -250 simply will not be drawn.

To fix our negative boundaries, we need to push everything to the right and down. If xmin is -250, we need to shift all pixels to the right by 250. We calculate our shifts by taking the negative value of our minimums.

shift_x = -xmin
shift_y = -ymin

print(f"Shift X: {shift_x}, Shift Y: {shift_y}")

Output:
Shift X: 250, Shift Y: 50

To tell OpenCV to apply this shift, we create a simple 3x3 mathematical grid called a translation matrix. In computer graphics, a translation matrix looks like this:

translation = np.array(
    [
        [1, 0, shift_x],
        [0, 1, shift_y],
        [0, 0, 1],
    ],
    dtype=np.float64,
)

The 1s on the diagonal keep the image at its original size, while shift_x and shift_y tell the computer exactly how many pixels to slide the image over.

We now know how large our canvas needs to be, and we have our translation matrix ready. Let's calculate the final width and height of our new, expanded canvas.

canvas_width = xmax - xmin
canvas_height = ymax - ymin
size = (canvas_width, canvas_height)

Now, it is finally time to move the pixels of our source image. We will use cv2.warpPerspective, which takes the image, the mathematical rule, and the final canvas size.

However, we do not just want to warp the image; we also want to shift it so it does not get cut off. We can combine our stretch (the homography) and our shift (the translation matrix) by multiplying them together using Python's matrix multiplication operator, @.

# Apply the shift and the warp in one single step
warped_source = cv2.warpPerspective(source, translation @ homography, size)

The warped_source variable now holds our left image, perfectly stretched to align with the right image and pushed safely into positive coordinates so that nothing is cut off!

We have successfully warped our source image onto the expanded canvas, but our target image is currently missing. We need to place it onto the same canvas so we can see how they line up.

First, we create a completely black, blank canvas that is the exact same size as our warped_source image.

# Create a black canvas matching the size and data type of the warped source
target_canvas = np.zeros_like(warped_source)

Because we shifted our source image by shift_x and shift_y, the target image needs to be shifted by the exact same amount to stay aligned. We can place the target image onto the black canvas using standard Python array slicing.

# Insert the target image pixels into the shifted location
target_canvas[shift_y : shift_y + h2, shift_x : shift_x + w2] = target

Finally, to verify our alignment, we can blend the two images together. We will use cv2.addWeighted, which acts like stacking two transparent slides on top of each other.

# Blend the warped source (50% visibility) with the target canvas (50% visibility)
preview = cv2.addWeighted(warped_source, 0.5, target_canvas, 0.5, 0)

The output preview will show both images overlapping. If our homography and our warping math are correct, the overlapping features (like the corners of buildings or the edges of mountains) will line up perfectly!

Excellent work! You have just learned how to safely warp an image without losing any pixels. Let's briefly recap the steps you took:

1. Found the Boundaries: You warped the corners of the source image to see where it would land and calculated the minimum and maximum coordinates.
2. Created a Translation Matrix: You determined how far the image needed to shift to avoid negative coordinates.
3. Warped the Image: You combined the translation shift with the homography stretch to warp the source image onto a larger canvas.
4. Blended the Results: You placed the target image onto the same expanded canvas and blended them to see the final alignment.

You are now well on your way to building a complete panorama stitcher in Unit 4! In the upcoming practice exercises, you will jump into the CodeSignal IDE to write these boundary calculations, translation matrices, and warp functions yourself. Let's get to coding!