Multimodal Data Sets

Core Datasets and Research Opportunities

Center for Data Science in Humanities

Chosun University

1. Infant-Perspective Visual Experience and Multimodal Learning

BabyView Korea captures infant-perspective video via head-mounted cameras — what infants see, how people and objects appear around them, and how that visual input may shape early learning. Collection began in 2025 and is ongoing. Relevant approaches include computer vision, multimodal analysis, behavioral coding, and computational modeling.

2. Bilingual LENA and Daylong Audio Corpora

Since 2025, the center has been collecting daylong audio from bilingual families in Korea using LENA, capturing bilingual exposure, caregiver-child interaction, and everyday language use. This builds on an established monolingual LENA corpus from earlier work. The same families also participate in remote looking-while-listening (LWL) studies via eBabyLab, linking naturalistic audio to controlled measures of real-time language processing. Opportunities span speech annotation, acoustic analysis, NLP, and computational modeling of bilingual speech environments.

3. Elderly Daylong Speech, PET Imaging, and Cognitive Aging

This dataset is complete: daylong speech from 120 older adults across CN− (cognitively normal, amyloid-negative), CN+ (cognitively normal, amyloid-positive), and MCI+ (mild cognitive impairment, amyloid-positive) groups, paired with PET imaging and cognitive testing; a subset also includes plasma p-tau217 measures. It supports research treating speech as a marker of cognitive and neurobiological change, through acoustic analysis, NLP, or predictive modeling.

Additional Experimental Research

The center also runs experimental studies of infant and child language processing (HPP, IPLP, looking-while-listening), including remote LWL via eBabyLab. This includes ManyBabies at Home – LWL, a multi-lab project led by the center, examining speech perception, word recognition, and bilingual exposure — a controlled complement to the naturalistic data above.