`Illumination Changes Dawn till Dusk (ICDD)', contains time lapsed 20K video frames for training (`video001' and `video003') and 1265 frames for testing (`video002' and `video004'). The dataset consisted of two different views, where `video001' and `video002' are in view 1, and `video003' and `video004' are in view 2. In this dataset, the spatial resolution of the videos is 720 x 576. The video sequences (`video001' and `video003') contain only dynamic background information with varying illumination conditions and were used for training purposes. The other two video sequences (i.e., `video002' and `video004') contained the background and moving objects and were used for testing purposes. The training video sequences were captured from dawn until dusk. Each spanned almost 12 h, with a time-lapsed condition to collect the background frames with many illumination conditions. The testing sequences contained moving objects with similar challenges.