This project implements an end-to-end panoramic image stitching pipeline similar to smartphone panorama modes. The system detects visual features across overlapping images, matches them, estimates geometric transformations, and blends images into a final panorama.

The goal was to design a modular computer vision pipeline capable of processing raw image sequences into a seamless stitched panorama using Python, OpenCV, and NumPy.

The pipeline follows a structured multi-stage process:

1. Corner Detection

2. Adaptive Non-Maximal Suppression (ANMS)

3. Feature Descriptor Generation

4. Feature Matching

5. RANSAC Homography Estimation

6. Image Warping & Blending

7. Multi-Image Panorama Stitching

The first stage identifies strong keypoints in each image using Harris and Shi-Tomasi corner detection. After testing both, Shi-Tomasi produced more evenly distributed and reliable features.

Approach:

• Convert images to grayscale

• Detect corners with OpenCV

• Visualize detected interest points

Python:

Result:

To prevent feature clustering and ensure strong spatial coverage, ANMS was applied to filter and distribute corners more evenly across the image.

A KD-Tree structure was used to accelerate nearest-neighbor searches, significantly improving runtime performance compared to brute-force filtering.

Goal: Keep the strongest corners while maintaining spatial diversity.

Each retained keypoint is encoded as a compact descriptor representing the local image region.

Process:

• Extract a 40×40 image patch centered on each keypoint

• Apply Gaussian blur

• Downsample to 8×8

• Flatten into a 64-dimensional descriptor

• Normalize for illumination invariance

This produces robust descriptors suitable for cross-image matching.

Python:

Descriptors are compared across images using Sum of Squared Differences (SSD). A ratio test filters ambiguous matches to retain only high-confidence correspondences.

Matching Strategy:

• Compute SSD across descriptor pairs

• Select best and second-best matches

• Apply Lowe-style ratio threshold

• Visualize match lines between images

Python:

Result:

To remove incorrect matches and estimate a robust homography, a custom RANSAC (Random Sample Consensus) implementation was applied.

Method:

• Randomly sample 4 matched keypoints

• Compute candidate homography

• Project points and compute SSD error

• Identify inliers below error threshold

• Select homography with maximum inliers

• Re-estimate homography using inlier set

This ensures transformation estimation remains stable even with noisy matches.

Python:

Estimated transformation matrices were made from real image pairs.

First matrix:

[[ 5.38576198e-01 -2.34639596e-01  9.60681637e+01]

 [ 7.32633209e-03  4.73562806e-01 -9.53412762e+01]

 [-4.41585762e-06 -1.07649918e-03  1.00000000e+00]]

Second matrix:

[[ 1.25207035e+00  1.36210576e-01 -7.32331499e+01]

 [ 4.24242072e-02  1.21663376e+00 -2.91451405e+02]

 [ 2.24391727e-04  3.48851141e-04  1.00000000e+00]]

Result:

Using the computed homography, one image is warped into the coordinate space of the other using cv2.warpPerspective. Images are then blended to create a smoother transition.

Challenges addressed:

• Exposure mismatch

• Photometric differences

• Edge seam artifacts

A weighted alpha blend was used, with optional support for more advanced blending (e.g., Poisson blending).

Python:

The full pipeline is extended to stitch multiple images sequentially into a single panorama.

Process:

• Initialize panorama with first image

• Detect features in current panorama and next image

• Apply ANMS

• Generate descriptors

• Match features

• Estimate homography

• Warp and blend

• Repeat for remaining images

Python:

Key Engineering Challenges & Solutions

• Compared Harris vs. Shi-Tomasi corner detection and selected more stable features

• Optimized ANMS using KD-Tree acceleration

• Resolved descriptor-corner alignment bugs affecting match accuracy

• Reduced false matches using RANSAC outlier rejection

• Generated robust homographies for image warping

• Successfully stitched multiple images into a unified panorama

Tools & Technologies

• Python

• OpenCV

• NumPy

• SciPy

• Matplotlib

• Jupyter Notebook

• Computer Vision

• Image Geometry

• Feature Matching