EventSleep: Sleep Activity Recognition
with Event Cameras

ECCVW 2024 - CVIU 2026

Carlos Plou, Nerea Gallego, Miguel Marcos, Alberto Sabater, Eduardo Montijano, Pablo Urcola, Luis Montesano, Rubén Martinez-Cantin, Ana C. Murillo

RoPert, DIIS - I3A, University of Zaragoza Bitbrain Technologies, Zaragoza, Spain

[Dataset] [EventSleep1-Paper] [EventSleep1-Code] [EventSleep2-Paper] [EventSleep2-Code]

Project Context

DeepSleep is a project in collaboration between the RoPert research lab from the University of Zaragoza, and BitBrain, a neurotechnology company that combines neuroscience, artificial intelligence, and hardware to help research, tech and health professionals to leverage neuroscience in a practical and reliable way.
As part of a broader study centered on sleep, this project investigates the suitability of event cameras, to analyze in a non-invasive manner specific behaviors that occur while sleeping and lead to sleeping disorders.

Introduction

Event cameras are a promising technology for activity recognition in dark environments due to their unique properties. However, real event camera datasets under low-lighting conditions are still scarce, which also limits the number of approaches to solve these kind of problems, hindering the potential of this technology in many applications. We present EventSleep, a new dataset and methodology to address this gap and study the suitability of event cameras for a very relevant medical application: sleep monitoring for sleep disorders analysis.

EventSleep1

The EventSleep1 dataset is composed of video sequences of common sleeping movements, recorded with an infra-red and a event camera. The use of a realistic recording scenario makes EventSleep the first event-based dataset of activity recognition that includes event sequences recorded under darkness. EventSleep dataset is already available at Synapse, however infrared data access requires a formal request through email. Further details can be found in the Wiki Synapse page (Section: To request Infrared data access).

The EventSleep1-data includes video sequences of14 different participants, 3 different set up configurations that include different ligthing conditions and occlusions, and 10 different fine-grained classes.

Cameras

Trials were recorded by two different cameras: Event (DVXplorer camera, 640 x 480 resolution) and Infrared (ELP HD Digital Camera, 6fps). Both cameras were placed together, attached to a metal bar, facing down and located 2 meters above the center of the bed, ensuring a full observation of the participants movements.

Scene Configurations

EventSleep was recorded in a room that mimics a regular room with a double-size bed. Each participant was recorded under 3 settings with varying conditions:

Config1: The subject is covered by an eiderdown under full darkness.

Config2: The subject is covered by an eiderdown under partial darkness.
Config3: The subject is uncovered under full darkness.

The full darkness (≤0.1Lux) setup consists of a bedroom with all lights off, door and window closed, with dense blinds covering the window. The partial darkness (0.2Lux) setup consists of a small night lamp on, placed on the floor far away from the head of the subject.

Activity Labels

All subjects were instructed to execute a certain set of movements and positions during each trial. These movements were carefully selected based on their potential relevance to medical research as part of a larger biomedical study project.

Within this context, experts in the field identified specific actions that appear in normal sleep (e.g. transitions between sleep positions) and in sleep disorders such as restless legs syndrome or periodic limb movement (e.g. shaking the legs). Since many sleep studies require the recording of physiological information, we also considered positions that could impact the performance of other sensors such as a headband for polysomnography. The final set of 10 activity labels considered is:

Demos

ResNet-IR vs ResNet-E

ResNetIR-ResNetE.mp4

ResNet-BE

s9_c1_bayesian_slow.mp4

EventSleep2

EventSleep2 data, an extension of the EventSleep dataset, that includes ten sleep recordings, around 7-hour long, six of which are synchronized with EEG data captured from a neuroheadband. This provides unique new data for the sleep research community, in particular for the promising event camera based application in this field. EventSleep2 dataset is also available at Synapse.

Citation

@inproceedings{plou2024eventsleep, title={EventSleep: Sleep Activity Recognition with Event Cameras}, author={Plou, Carlos and Gallego, Nerea and Sabater, Alberto and Urcola, Pablo and Montijano, Eduardo and Montesano, Luis and Martinez-Cantin, Ruben and Murillo, Ana C}, booktitle={Computer Vision -- ECCV 2024 Workshops}, pages={52--69}, year={2025}, organization={Springer}}
@article{gallego2026eventsleep2, title={EventSleep2: Sleep activity recognition on complete night sleep recordings with an event camera}, author={Gallego, Nerea and Plou, Carlos and Marcos, Miguel and Urcola, Pablo and Montesano, Luis and Montijano, Eduardo and Martinez-Cantin, Ruben and Murillo, Ana C}, journal={Computer Vision and Image Understanding}, pages={104619}, year={2026}, publisher={Elsevier}}

Acknowledgments

This work was supported by PID2024-159284NB-I00, PID2021- 125514NB-I00, PID2024-158322OB-I00, PID2021-125209OB-I00, and AIA2025-1635 grants funded by MCIN/AEI/10.13039/501100011033 ERDF/NextGenerationEU/PRTR, grant no. 101135782 (MANOLO project) funded by the European Union, two DGA scholarships and project T45_23R.

Page updated

Report abuse