Small Contributions, Small Networks: Efficient Neural Network Pruning Based on Relative Importance

Mostafa Hussien, Mahmoud Afifi, Kim Khoa Nguyen, and Mohamed Cheriet 

arXiv, 2024

In collaboration with ÉTS, University of Quebec

We introduce a data-driven unstructured pruning method that uses training data (or a subset of it) to estimate each weight's importance based on its contribution to the corresponding node's output. Preliminary experiments show that our method consistently produces more accurate models, even at high compression rates, compared to other approaches.

arXiv 

What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models

Abdelrahman Abdelhamed*, Mahmoud Afifi*, and Alec Go

arXiv, 2024

Google

With some prompt engineering, multimodal LLMs (e.g., Gemini) can perform zero-shot image classification. However, they may not consistently produce accurate target dataset labels. Our approach leverages multimodal LLMs & cross-modal embedding encoders to produce initial class prediction feature & image description feature alongside image feature, improving zero-shot image classification accuracy without the need for dataset-specific prompts. Our method outperforms prior methods across various datasets, achieving a 6.8% increase in accuracy on ImageNet.

* Equal contribution 

arXiv | Code & Data

Optimizing Illuminant Estimation in Dual-Exposure HDR Imaging

Mahmoud Afifi, Zhenhua Hu, and Liang Liang

ECCV 2024

Google

Utilizing the chromatic distortion present between long and short exposure frames of HDR photography, we introduce a compact guiding feature for illuminant estimators. Processed by just ~300 learnable parameters, it achieves results that match or surpass previous methods relying on thousands or even millions of parameters.

Paper | Supp. Materials | arXiv | Poster

Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs

Georgy Perevozchikov, Nancy Mehta, Mahmoud Afifi, and Radu Timofte

ECCV 2024

In collaboration with University of Würzburg

Rawformer, an unsupervised Transformer-based encoder-decoder model for raw-to-raw mapping, enables the utilization of learnable camera ISP trained on a specific camera's raw images to render raw images taken by new cameras with different characteristics.

Paper | Supp. Materials | arXiv | Code

Auto White-Balance Correction for Mixed-Illuminant Scenes

Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown

WACV 2022

York University 

Mixed/single-illuminant scene white balancing does not necessarily require illuminant estimation. Instead, the problem could be bounded by a small set of predefined white-balance settings. Given that, we locally blend a set of small images rendered with different white-balance settings to generate the final corrected image.

Paper | Supp. Materials | arXiv | Code & Data | Poster | Presentation | Talk | Patent Application

Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning

Abdullah Abuolaim, Mahmoud Afifi, and Michael S. Brown

WACV 2022

York University 

Jointly learning to predict the two DP views from a single blurry input image improves the network’s ability to learn to deblur the image. Generating high-quality DP views can be used for other DP-based applications, such as reflection removal.

PDF | Supplementary Materials | arXiv | Code & Dataset | Patent Application

Cross-Camera Convolutional Color Constancy

Mahmoud Afifi, Jonathan T. Barron, Chloe LeGendre, Yun-Ta Tsai, and Francois Bleibel  

ICCV 2021 (Oral Presentation)

Google Research

A self-calibration method for cross-camera color constancy through the lens of transductive inference: additional (unlabeled) images are provided as input to the model at test time, which allows the model to calibrate itself to the spectral properties of the test-set camera during inference.

Paper | Supp. Materials | arXiv | Code | Poster | Presentation | Presentation (PDF) | Talk | Google @ ICCV'21

HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms

Mahmoud Afifi, Marcus A. Brubaker, and Michael S. Brown

CVPR 2021 

York University 

HistoGAN is the first work to control colors of GAN-generated images based on features derived directly from color histograms. Our method learns to transfer the color information encapsulated in histogram features to the colors of a GAN-generated images (HistoGAN) or real input images (ReHistoGAN). As color histograms provide an abstract representation of image color that is decoupled from spatial information, our HistoGAN and ReHistoGAN are less restrictive and suitable across arbitrary domains.

Paper | Supp. Materials | arXiv | Code & Dataset | Colab (histogram loss) | Poster | Presentation | Talk

Learning Multi-Scale Photo Exposure Correction

Mahmoud Afifi, Konstantinos G. Derpanis, Björn Ommer,  and Michael S. Brown

CVPR 2021 

Samsung Research in collaboration with Heidelberg University

A single coarse-to-fine deep learning model with adversarial training to correct both over- and under-exposed photographs.

Paper | Supp. Materials | arXiv | Code & Dataset | Poster | Presentation | Talk | Samsung Research Post | Patent

CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

Mahmoud Afifi, Abdelrahman Abdelhamed, Abdullah Abuolaim, Abhijith Punnappurath, and Michael S. Brown

TPAMI 2021

York University 

Learning an accurate camera-rendering linearization gives a significant improvement for different computer vision tasks (e.g., denoising, deblurring, and image enhancement).

Paper | arXiv | Supp. Materials | External Link | Code & Dataset 

Semi-Supervised Raw-to-Raw Mapping

Mahmoud Afifi and Abdullah Abuolaim

BMVC 2021

York University 

A semi-supervised method to map between two different camera-raw spaces. Training requires an unpaired set of images besides a very small set of paired images taken by these two camera models. 

arXiv | Dataset | Presentation 

Deep White-Balance Editing

Mahmoud Afifi and Michael S. Brown

CVPR 2020 (Oral Presentation)

Samsung Research

A multi-task deep learning model for post-capture white-balance correction and editing

Paper | Supp. Materials | arXiv | Code | Presentation | Talk | CVPR Daily Magazine | Samsung Research Post | Samsung Newsroom | Patent

Modeling Defocus-Disparity in Dual-Pixel Sensors

Abhijith Punnappurath, Abdullah Abuolaim*, Mahmoud Afifi*, and Michael S. Brown

ICCP 2020

York University 

A symmetry property of dual-pixel kernels for unsupervised depth estimation

* Equal contribution 

Paper | Code & Dataset | Talk

System and Method for Color Matching

Atima Lui, Nyalia Lui, Mahmoud Afifi, and Ariadne Bazigos

US Patent 2020

My Nudest Inc

A system for analyzing user input, combining user's image(s) and query responses to provide tailored color outputs. Through color correction and comparison to predetermined color identifiers, it delivers accurate results and product recommendations.

Patent | PDF

AIM 2020: Scene Relighting and Illumination Estimation Challenge

Majed El Helou, Ruofan Zhou, Sabine Süsstrunk, Radu Timofte, Mahmoud Afifi, et al.,

ECCV workshops 2020

York University

Runner-Up Award overall tracks

As an intermediate stage, producing a uniformly-lit white-balanced image could help to eventually produce high-quality relit images.

arXiv | Code | Certificate

Interactive White Balancing for Camera-Rendered Images

Mahmoud Afifi and Michael S. Brown

CIC 2020 (Oral Presentation)

York University 

A simple method to link the nonlinear white-balance correction functions, introduced in our CVPR'19 work, to the user's selected colors to allow interactive white-balance manipulation

arXiv | Code | Presentation

Improving Color Space Conversion for Camera-Captured Images via Wide-Gamut Metadata

Hoang Le, Mahmoud Afifi, and Michael S. Brown

CIC 2020 (Oral Presentation)

York University 

Having additional wide-gamut metadata, available during color space conversion, greatly assists in constructing a locally weighted color mapping function to convert between color gamuts. 

Paper | Code & Dataset

What Else Can Fool Deep Learning? Addressing Color Constancy Errors on Deep Neural Network Performance

Mahmoud Afifi and Michael S. Brown 

ICCV 2019 

York University 

Deep learning models can be fooled by white-balance errors and their accuracy can be improved by augmented images with different white-balance settings.

Paper | Supp. Materials | arXiv | Project Page | Code | Colab

When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown 

CVPR 2019

York University in collaboration with Adobe Research

The first work to directly address the problem of incorrectly white-balanced images; requires a small memory overhead and it is fast.

Paper | Project Page | Supp. Materials | Demo | Video | Code | Dataset | Adobe Research Post | Patent

Image Recoloring Based on Object Color Distributions

Mahmoud Afifi, Brian Price, Scott Cohen, and Michael S. Brown.

Eurographics 2019 (Short Papers)

York University in collaboration with Adobe Research

A fully automated image recoloring without the need for target/reference images

Paper | Project Page | External Link | Code | Supp. Materials | Presentation | Fast-Forward Video | Adobe Research Post

Sensor-Independent Illumination Estimation for DNN Models 

Mahmoud Afifi and Michael S. Brown 

BMVC 2019 (Oral Presentation)

York University 

Learning a new canonical space in an unsupervised manner allows us to train a single deep model on multiple camera sensors and perform accurate illuminant estimation for images captured by new unseen camera sensors in the inference phase.

Paper | Project Page | Supp. Materials | arxiv | Code | Presentation | Talk | Patent

Color Temperature Tuning: Allowing Accurate Post-Capture White-Balance Editing 

Mahmoud Afifi, Abhijith Punnappurath, Abdelrahman Abdelhamed, Hakki Can Karaimer, Abdullah Abuolaim, and Michael S. Brown 

CIC 2019 (Oral Presentation)

Best paper award

York University 

With a small modification to existing camera ISPs, we can achieve accurate post-capture white balance editing by embedding a set of mapping coefficients in the JPEG metadata.

Paper | Project Page | Code | Presentation | Patent Application

As-Projective-As-Possible Bias Correction for Illumination Estimation Algorithms

Mahmoud Afifi, Abhijith Punnappurath, Graham Finlayson, and Michael S. Brown 

JOSA A 2019

York University in collaboration with University of East Anglia

A locally adaptive bias correction technique for illuminant estimation

Paper | Project Page | External Link | Code 

Tensor Methods for Group Pattern Discovery of Pedestrian Trajectories

Abdullah Sawas*, Abdullah Abuolaim*, Mahmoud Afifi, and Manos Papagelis

MDM 2018

Best paper award

York University 

Efficient discovery of evolving groups of pedestrians; a new group pattern is introduced in the journal version.

* Equal contribution

Project Page | Poster | Journal Version

AFIF4: Deep Gender Classification Based on AdaBoost-Based Fusion of Isolated Facial Features and Foggy Faces

Mahmoud Afifi and Abdelrahman Abdelhamed 

JVCI 2019

Assiut University 

Gender classification can be improved using different facial features; our finding is validated by a user study.

arxiv | Dataset

11K Hands: Gender Recognition and Biometric Identification Using a Large Dataset of Hand Images

Mahmoud Afifi

MTA 2019

Assiut University 

Hand images can efficacy be used for gender recognition and biometric identification; a large dataset of hand images enables us to train our two-stream deep model.

arxiv | Project Page & Dataset | Code

A Comprehensive Study of the Effect of Spatial Resolution and Color of Digital Images on Vehicle Classification

Khaled F. Hussain, Mahmoud Afifi, Ghada Moussa 

ITS 2018

Assiut University 

In generic vehicle classification tasks, deep learning models tend to not rely on color information and filter out unnecessary details information in high-resolution images.

Paper | External Link

MPB: A Modified Poisson Blending Technique

Mahmoud Afifi and Khaled F. Hussain

CVM 2016

Assiut University 

Bleeding artifacts caused by Poisson image editing can be reduced by a simple two-stage blending approach.

Paper | Project Page | Code | Video

Conference version: Paper | Project Page | Code | Video

Fast Video Completion Using Patch-Based Synthesis and Image Registration

Mahmoud Afifi, Khaled F. Hussain, Hosny M. Ibrahim, and Nagwa M. Omar

ISPACS 2015

Assiut University 

In many scenarios, image registration and blending can help to get a fast video completion.

External Link | Code