Workshop on Multi-Biological Sensing Data for Speech and Language Deterioration Prediction

In conjunction with ACM Multimedia Asia 2024

Photo copyright @ Central Vermont Medical Center, University of Vermont

SpandLDeteriorate Workshop of ACM Multimedia Asia 2024

(Dec. 3th afternoon, Auckland, New Zealand)

The workshop will be held in a hybrid format. Join the online workshop by PC, Mac, iOS or Android here.

Meeting ID: 812 9981 5991

Password: 184575

Overview

Digital health applications are a hot topic in Computer Vision, NLP, Audio Analysis, and Biomedical Informatics, with significant potential impact across various domains, including the prediction of language deterioration. However, researchers still lack a comprehensive understanding of multimodal biological analysis for language deterioration. The relationship and dynamics between different sensor data and speech data are particularly understudied, especially in longitudinal studies and across heterogeneous populations. With the advent of large pre-trained models, particularly multimodal LLMs, it has become increasingly feasible to enhance language deterioration detection using diverse data modalities. Simultaneously, further research into assistive technologies is essential to improve speech communication for patients with speech and language disorders, thereby enhancing communication efficiency and monitoring mental health. SPandLDeteriorate is thus a significant addition to ACM Multimedia Asia 2024, offering a unique platform to discuss the convergence of these critical areas.

Guideline

Submission: Papers on SpandLDeteriorate can be submitted to the workshop through the ACM Multimedia Asia 2024 author console of paper management system: https://cmt3.research.microsoft.com/MMAsia2024.

Paper format: Submitted workshop papers basically follow the ACM Multimedia Asia-2024 short paper style, format, and 2-6 pages (Paper Submission Guidelines).

In-person or online: The Satellite Workshops will be held with in-person (online just in case) attendance. Accordingly, each accepted workshop paper must be presented in-person (online just for personal health, VISA, or national restrictions) by one of the authors.

Registration: The main conference will provide the registration with the official entry of the workshop paper. One main conference paper can cover one workshop paper. Note that if the authors of workshop papers would like to attend the main conference, it requires the main conference registration.

Publication: Upon acceptance, paper authors will have the opportunity to present their paper at our workshop, and the workshop paper will be included in the workshop proceedings belonging to the ACM MMAsia2024.

Anonymity Policy: Both submission types will be single-blind, to prevent authors from having to spend too much time removing all references to the student demographic, location, university etc.

Best paper award: We will select the best paper among the accepted submissions for an award during the workshop.

Topics

We invite submissions of original technical papers related to the SpandLDeteriorate (topic of this workshop) including but not limited to:

Multimodal fusion for speech disease detection
Speech disease detection in the wild
Machine learning methods for language deterioration detection
LLMs and foundation models for speech disease detection
LLMs and foundation models for dementia EHR analysis
Speech, voice and hearing disorders
Pathological speech recognition
Information retrieval models for chronic disease
Biosignal-based speech recognition and synthesis
Longitudinal and remote data collection and analysis
Feature extraction and novel representations that provide clinical interpretability

Important Dates

Workshop Paper Submission Deadline: Sept. 27. 2024 - extended to Oct. 27. 2024
Workshop Paper Acceptance Notification: Oct. 11. 2024 - extended to Nov. 10. 2024

Workshop Schedule

13:45-14:00 Opening

14:00-14:45 Keynote 1: Challenges in Clinical Natural Language Processing (Dr. Mike Conway)

14:45-15:00 Paper Presentation: Reference-free automatic speech severity evaluation using acoustic unit language modelling (Bence M Halpern)

15:00-15:15 Paper Presentation: Free-FreeSLT: A Gloss-Free, Parameter-Free model for Sign Language Translation (Weirong Sun)

15:15-15:30 Break and Afternoon Tea

15:30-16:15 Keynote 2: Research on Automatic Detection of Alzheimer's from Speech: Progress and Reflections (Dr. Jiahong Yuan)

16:15-16:30 Paper Presentation: Swin-BERT: A Feature Fusion System designed for Speech-based Alzheimer's Dementia Detection (Yilin Pan)

16:30-16:45 Close

Keynote Speakers

Jiahong Yuan

University of Science and Technology of China, China

(jiahongyuan-at-ustc.edu.cn)

Research on Automatic Detection of Alzheimer's from Speech: Progress and Reflections

Abstract: Language impairments in Alzheimer's Disease (AD) manifest across various levels of linguistic structure. Leveraging the Transformer model and the pretraining-finetuning approach, machine learning can effectively capture and utilize these features for automatic AD recognition.

Our research demonstrates that incoporating pause encoding in word transcription, combied with pre-trained language models that integrate pause information, significantly enhances the accuracy of automatic AD recognition. This talk presents thse findings and explores the potential of contextual pauses as a biomarker for AD. Additionally, it will discuss the challenges and strategies involved in constructing speech datasets for AD detection.

Mike Conway

The University of Melbourne, Australia (mike.conway-at-unimelb.edu.au)

Challenges in Clinical Natural Language Processing

Abstract: The application of Natural Language Processing (NLP) methods to real-world clinical text data derived from Electronic Health Records has the potential to improve quality of patient care, enhance the efficiency of healthcare systems, and support clinical and public health research. However, working with clinical text is challenging.

In this talk I will attempt to do four things. First, I will describe the broad terrain of clinical NLP, including applications in clinical decision support and epidemiology research. Second, I will outline some of the distinctive challenges involved in developing clinical NLP algorithms. Third, I will describe current methods and resources used in clinical NLP applications. Finally, I will present the argument that -- at least as things stand in late 2024 -- Large Language Models are often not well suited for some clinical NLP tasks.

Organizers

For questions, please contact organizers at: yilin.pan@dlmu.edu.cn or other organizers.