Gigapixel Panorama Acquisition and Stitching Workflow

Image Acquisition, Processing and Stitching Workflow for Gigapixel Panoramas

General Considerations

• The camera: I preferred bridge cameras like  Panasonic Lumix FZ28 or the Lumix FZ200 for a long time. They are light, have a large zoom range and a decent image quality.  However the small sensor size covers only a tiny field.  With a full-frame camera you can acquire the same panorama with fewer images in shorter times! Plus -  the images of a full-frame mirrorless camera  contain less sensor noise.

• Provide plenty of image overlap (30-40 %). Sufficient overlap is helpful  in case of positioning errors or lens vignetting. 

• Light intensity shouldn't change too much during image acquisition . Starting  the acquisition of a very large panorama half an hour before sunset will certainly result in huge intensity gradients! Large intensity variation will  occur if moving clouds cast their shadows in the scenery.

Camera Settings

• Acquire raw mages (rather than jpg or tiff). Increased bit depth (typically 10-14 bit) increases the dynamic range, improves image quality and reduces  panorama artifacts (banding etc.)

• Shoot images in Manual Mode and use the same camera settings throughout the complete panorama acquisition

• Typical settings I use with my A7R II and a Minolta AF 100-400mm F4.5-6.7 APO telezoom lens :  f/11, f=400mm, exposure 1/400 s, ISO 100 or 200. If you use a full-frame sensor with a long focal length objective (e.g. 400 mm) make sure to  close down the aperture to f/10 or more. This is necessary to minimize vignetting. Even if your software has vignetting correction , often banding artifacts remain - especially where you expect to see a homogeneous sky.

• Minimize sensor noise by using a low ISO value.  However, when using image processing software with good AI based noise reduction algorithms (e.g. DxO Photolab) you can use higher ISO values (e.g. ISO 500 or above) and reduce the exposure time.

• Silent Shutter is a nice option if you want to save your cameras shutter mechanics from premature wearout, however, having in mind the rolling shutter effect, you should ensure that the camera isn't shaking during exposure. Be careful using Electronic First Curtain Shutter (EFCS), short exposure times can possibly result in intensity gradients. These aren't noticeable in the single images, but they can spoil your panorama (vertical banding).

• Avoid using exposures shorter than 1/600 s (too short exposures can possibly result in intensity variations and gradients)

• Find a good focus using the autofocus and then  keep the setting in manual mode

Settings for your automated panorama rig:  wait long enough to make sure mechanical jitter has stopped before triggering

The acquisition rate of my setup is currently limited to about 2-3 s per image. This time is required for saving large raw images to the cameras SD card.

Raw Image Processing 

For raw image processing I am using    DxO Photolab 7, and sometimes Capture One.

For the panorama stitching software PTGUI 16bit input is recommended - not only to increase the dynamic range  - but also to avoid banding artifacts in plain colored parts of the panorama (e.g. blue sky).

• In order to reduce blur (e.g. in windy weather) you can reduce the exposure time to 1/500s or shorter.  But then my images will be noisy! Not much of a problem with DxO Photolab 7 and   Deep Prime  (AI based image enhancement and noise reduction)

• DxO ClearView Plus is a very nice feature  for removing atmospheric haze, thus greatly increasing contrast of distant features.  However, I found that it seems  to introduce a kind of artificial lens vignetting. Neither DxOs vignetting correction nor PTGuis vignetting correction can remove these  artifacts, which are showing as a repetitive pattern in the skies.  As a remedy, if you provide sufficient  image overlap (e.g. 40%),  you can apply cropping (in PTGui) to significantly reduce these artifacts. 

• dust spots on the sensor are currently my biggest nuisance. I still need a kind of automated spot removal in my workflow!

• correct chromatic aberration (doesn't make a difference with my gear and my standard settings)

• When using Capture One do not apply vignetting correction or lens corrections at this stage .
  When using DxO Photolab 7  applying the vignetting correction provides better results than that of PTGUI!

• Export  as lossless 16 bit tiff image

Tiff images will be about about 5 times larger than the raw images (41MB ->214MB   - and typically there are more than 500 of these!). For long term storage I keep the raw images and delete the processed images after stitching.

Gigapixel Panorama Stitching with PTGUI  Pro 12

• Load processed 16bit  tiff images with PTGUI Pro 12

• Arrange images in a regular grid or import Papywizard file containing individual image position data

• Level the panorama. For this purpose generate about 4 to 8 vertical control points. Sometimes Straightening the Panorama works also fine.

• Apply Exposure Fusion or Tone Mapping. 

• Exposure correction (fine tuning) and vignetting correction: sometimes it can be helpful if exposure correction is performed while vignetting correction is unchanged ("keep current").  Then,  do vignetting correction while leaving  the exposure correction unchanged. Repeat  until the panorama quality doesn't improve anymore.

• Identify images with missing control points (e.g. images containing a small portion of the horizon),  and add control points manually, optimize if necessary. Images without control points (e.g. showing sky only) can moved to the right position manually.

• Automatic exposure correction works best if "sky images" are placed correctly

• Manual fine tuning of individual image brightness (by setting the  exposure offset in the image parameters tab)

• Remove "ghosts" (e.g moving cars or people) by masking

Create panorama (stitching):

•  Choose output format:  .psb (photoshop big) - up to 300000 pixels in either dimension , tiff (size limit: 4 GByte) or jpg (size limit: 30000 pixels in either dimension)

• For export to Gigapan website use these output parameters: psb, compression: PackBits, Layers: flattened

Alternative - Create tiled panorama with krpanoTools and host in the cloud  on  Amazon's AWS S3 storage service

• Drag and Drop the large .psb image to "MAKE PANO MULTIRES" droplet. Don't use the "MAKE Pano FLAT" droplet,  as this will generate about 10 x more image tiles (=image files), and the subsequent upload to AWS S3 will take much longer.

• krpano creates a new folder containing image tiles html, xml and java scripts. Flash functionality is not necessary, and included only as a fallback solution.

• Tiled panoramas can be viewed by opening the index.html  with a web browser. Look and feel is similar like that of the Gigapan viewer.

• Another nice feature of krpano: the tiled panoramas are much smaller than the corresponding  .psb files

• Create an AWS account and  get a S3 storage.  Why AWS S3? Because they have no limits on file numbers.  The krpano output folder for one gigapixel panorama can easily contain 30000 image tiles...

• Uploading thousands of very small image tiles (512x512 pixels, ca. 64 kB)  to AWS is slow, due to a large overhead for each file. Upload rates are as slow as 30 kB/s .  Its more efficient to upload larger tiles (e.g. 2048x2048 pixels). In krpano the tile size can be set in the multires.config (in the  templates folder). 

Troubleshooting

Do you see vignetting artifacts in your panorama?

• Use Vignetting Correction in PTGUI.  Sometimes, however,  (depending on your camera hardware or camera settings) PTGUI doesn't correct vignetting perfectly. Then you may notice a regular pattern of bands or dots (or even rings) in your panorama, especially where you expect to see a homogeneously blue sky.

• Try Vignetting Correction in your photo editing software.

• Increase the image overlap

• Use cropping in PTGui, to get rid of gradients, which are typically strongest in the corners of the camera sensor.

• Reduce the lens aperture. I am mostly shooting at 400mm, and values of f/11 or above work fine. At f/8 I see artifacts, which cannot be corrected well.

• Do vignetting correction by using calibration images: Shoot a sequence of images showing the blue sky only. Use the same settings (also the same focus) like in your panorama. Create a calibration mask by averaging over the calibration images.  For this purpose I programmed a python script that does  a pretty good vignetting correction on 16-bit TIFF images. Sometimes it will work, sometimes it won't.

• Recently I got a Sigma 100-400 mm telephoto lens. A great lens, but even at f/13 I  couldn't get rid of the  vignetting artifacts with any of the above methods.  In absolute terms the vignetting is  small (at most a few percent intensity falloff at the corners), however,  artifacts are clearly visible in the sky regions of the panorama.  Maybe its the shape of the intensity falloff - almost perfectly constant in the center and then a sudden almost linear falloff towards the corners - that causes vignetting correction to fail.  How did I finally solve this problem?  At first I returned to  DxOs raw converter software.  PhotoLab 7  has a lens profile for my Sigma lens.  Additionally I increased the image overlap from 25% to 35%. That makes a big difference!  This seems to work fine, vignetting has gone.  More testing is necessary. 

• aggressive image processing (e.g. haze removal)  can introduce gradients (similar to vignetting) - try cropping 

How to automate your stitching workflow?
Most often PTGUI stitches even large panoramas (of 100s of images) very well -  not much manual adjustment necessary.
However,  sometimes the images do not stitch so well.  I found that a badly leveled tripod head often is the reason for such kind of troubles. Take time for leveling the tripod. Make sure that from the left edge to the right edge of your panoramas field of view the horizon is a the center of the middle image row.
In general sky images aren't stitched well, as they don't provide control points. Cloud structures could be used for finding control points. However, if the clouds are moving - and they almost always do, unless they are very far away -  it is not recommend.
If you shoot at long focal lengths, then foreground images may be blurred and therefore don't provide control points.
The usual workflow is to move these images (sky and foreground images) manually to their appropriate position.  I wrote a little python script that automates this work.

Panorama Stitching Machine

• Intel Core i7 CPU (9th gen)  8 cores 

• Gigabyte Z390 Mainboard

• Graphics Card: Nvidia RTX 3090

• Plenty of RAM (DDR4-3000 MHz) : 64 GB 

• Fast SSD:  1 TB Samsung 970 Evo  M2 SSD

On this machine stitching a  medium sized panorama (650 images ,10 rows, 65 columns, 5304x7952 pixels) takes about 2 hours, and results in a panorama size of 237875x54107 pixels [that was for the GTX 980] The psb file file size is  31 GByte.

Recently I replaced the Geforce GTX980 by a Geforce RTX3090.  With the RTX 3090  (and with PTGUI 12.13) stitching 395 Images (5304x7952 pixel) to a panorama with 227230x23426 pixels takes less than 20 minutes.

 Loading a PTGUI project with hundreds of images (processed tif images, 200MB each) from the SSD  is blazingly fast. Loading takes less than a minute.  And also the whole processing of the panorama  is really smooth now.