Results & Discussion

Results

Naive Sum + GAN also improves strongly over Naive Sum 16, especially in PSNR. However, it falls behind diffusion on MS-SSIM and LPIPS. This matches our qualitative observation that some GAN outputs contained flat or repeated noise artifacts, even when parts of the image were denoised well.

MCA alone performs competitively without using a learned refinement model. Its PSNR is close to the generative methods, and it improves substantially over Naive Sum 16 by using motion compensation before aggregation. However, its LPIPS is still worse than the diffusion-based methods, showing that alignment and aggregation alone do not fully recover perceptual detail.

MCA + Diffusion achieves the best MS-SSIM and the second-best PSNR. This suggests that using MCA as the Stage 1 input helps preserve structure before diffusion refinement. Its LPIPS is not as low as Naive Sum + Diffusion, so the improvement is strongest in structural fidelity rather than purely perceptual similarity.

ConvLSTM achieves the best PSNR and a strong MS-SSIM score, showing that learned temporal aggregation can recover pixel-level information very effectively from the full burst. Its LPIPS is higher than the diffusion-based methods, which suggests that the output is accurate in a pixel-wise sense but not always the most perceptually realistic.

Overall, the table shows a trade-off between pixel accuracy, structural similarity, and perceptual quality. While ConvLSTM is strongest for PSNR, meaning it has the highest pixel accuracy, MCA + Diffusion is strongest for MS-SSIM and LPIPS, making its produced images appear more realistic to the human eye. We can observe these differences in the qualitative results below.

Sample for Each Main Technique

Discussion

Insights from MCA

The main lesson from MCA is that motion estimation is only reliable once there is enough signal to align. Our initial frame-by-frame alignment approach was too sensitive to photon noise: individual binary SPAD frames contain so little structure that the estimated shifts often reflected noise rather than true camera motion. Aggregating nearby frames into small blocks before alignment made the process much more stable, because the block sums revealed edges and scene structure that were almost invisible in single frames.

We also found that the type of motion matters. Translation-only alignment works well for scenes with simple camera motion, but it is not enough when the burst contains zoom or mixed motion. Adding global scale estimation made the method more robust in these cases, and local TV-L1 optical flow helped refine residual motion that could not be captured by one global transform.

The main limitation of adaptive MCA is computational cost. Testing multiple zoom adjustments and applying dense local refinement is much slower than naive summation or simple blockwise translation. Future improvements would include extending MCA beyond the fixed 256-frame window when alignment remains reliable, making the overlap-based stopping rule more scene-dependent, and adapting the block size based on the amount of motion.

Insights from ConvLSTM

A simple ConvLSTM can learn to integrate 1,024 sparse binary frames far better than a raw average, showing that learned temporal aggregation is worth the effort even with a small model.​

Training at reduced resolution (400×400) and upsampling keeps GPU costs low while still producing usable reconstructions, making this practical on a single consumer GPU.​

Spatial structure matters. Because gates are convolutions (3×3), the network can use spatial context, if neighboring pixels have seen photons, that informs the current pixel's gate decisions. A non-convolutional LSTM cannot do this.

Chunking + state carry-over is practical. Processing 1,024 frames at once is impossible on one GPU. Chunking into 64-frame groups and carrying (h, c) across chunks lets you handle the full sequence with bounded memory while keeping the effective receptive field at 1,024 steps.

The whole architecture is 4 conv layers. Yet it produces a massive improvement. This suggests the temporal integration problem has a lot of low-hanging fruit that even a simple learned approach captures.

The model tends to predict a slightly smoothed version of the scene rather than committing to sharp edges, because the average error is lower that way. A perceptual loss would push it toward sharper outputs.

Single-layer ConvLSTM has limited temporal reasoning. With only one recurrent layer, the model mostly learns a weighted running average — it cannot do complex multi-step reasoning (e.g. explicit motion compensation across distant frames).

The ConvLSTM sees motion implicitly through the forget/input gates, but it does not align frames. Moving objects still get slightly blurred, less than naive sum, but not fully sharp.

Insights from Diffusion

Diffusion models are notoriously data-hungry, and at the scale of this competition's training set there was no realistic path to training a useful model from scratch. That is, leveraging a large pretrained diffusion backbone was a necessity rather than a convenience. Even with the pretrained prior in place, the latent-space MSE objective tended to over-smooth fine detail, since averaging across plausible high-frequency outcomes is the loss-minimizing behavior in latent space.

Adapting the pretrained model to our task also turned out to be harder than expected. LoRA-based fine-tuning was not powerful enough to redirect the model toward our conditioning signal. That is, the pretrained prior dominated and the model effectively ignored the input, hallucinating outputs unrelated to the measurement. Switching to selective unfreezing of the decoder gave the conditioning signal a real gradient path into the network and resolved the issue.

Looking ahead, the over-smoothing problem could be addressed at training time by adding LPIPS loss on the decoded prediction and Min-SNR loss weighting, both of which discourage averaging behavior. The dominance of the pretrained prior also points toward a backbone better matched to low-light photographic data, rather than the LAION-trained SD 2.1 we used. Finally, joint end-to-end training of the aggregation and diffusion stages could let the upstream pipeline learn to produce intermediates that the diffusion model handles more reliably.

Insights from GAN

We observe that some GAN-generated outputs contain uniform, flat noise patches superimposed on otherwise well-denoised regions, as shown below. These artifacts are spatially consistent and visually distinct from typical reconstruction noise, suggesting a systematic issue rather than random error.

One likely explanation is a resolution mismatch during training and inference. The model is trained on 256×256 crops to reduce computational cost, but evaluated on full 800×800 images. As a result, the U-Net may struggle to generalize to larger spatial contexts it has never seen during training. This can lead to patch-wise inconsistencies, where certain regions fail to integrate smoothly with surrounding structure.

While this hypothesis aligns with observed behavior, further experiments and full-resolution training are needed to confirm the underlying cause.