Emergent phenomena
in Deep Representations and Large Language Models
Emergent phenomena
in Deep Representations and Large Language Models
Deep learning models trained on large datasets have shown spectacular performance in a wide range of tasks demonstrated by current applications of Large Language Models. However, recent works have shown that the abilities large machine learning models acquire often emerge unpredictably with increasing model complexity or training dataset size. These emergent phenomena include the unexpected appearance of abilities for which the model was not explicitly trained, but they may also be related to unexpected performance boosts due to the increased model complexity. Emergent phenomena are not always beneficial: larger models may pick up new biases from the training data or start hallucinating.
To move towards increasingly sustainable, reliable, and explainable applications of AI systems, it is necessary to increase the understanding of the mechanisms surrounding emergent phenomena. Moreover, this effort provides increased insight into the learning process behind the acquisition of abilities of large models to perform specific tasks. Important research questions relate to the definition of emergent phenomena, their causes (what controls which abilities are acquired and when?), training efficiency, and training data quality (e.g., acquiring desired abilities with less computational effort), prompting strategies to get or test for desired model behaviour (e.g., a chain of thought), and further verification methods of model abilities and properties.
The primary goal of this special session is (i) to discuss the emergent abilities and risks in deep neural networks and representations from very different angles and (ii) facilitate networking and encourage collaboration between various research fields that approach this issue from different perspectives, like computational linguistics, ethics in AI, computer science, physics, etc.
Topics of interest include, but are not limited to:
The definition of emergence in the context of NLP and ML
Prompting strategies
Physics-based/inspired analyses (e.g. phase transitions in ML models)
Explainability and interpretability (XAI)
Evaluation measures for model ability, monitoring strategies, assessment of model abilities (e.g. technical or psychology-based)
Knowledge distillation, model pruning, energy-efficient models.
Mitigation strategies for emergent risks and model deterioration.
Fine-tuning and Retrieval-augmented generation (RAG)
Papers focusing on specific emergent phenomena (reasoning, creativity, Double descent phenomena etc.
Organizing Committee:
Dr. Özge Alacam (Ludwig-Maximilian University of Munich & Uni Bielefeld, Germany)
Dr. Michiel Straat (Uni Bielefeld, Germany)
Prof. Dr. Hinrich Schütze (Ludwig-Maximilian University of Munich, Germany)
Prof. Dr. Alessandro Sperduti (University of Padova, Italy)
Important Dates: (Anytime-on-Earth):
Paper Submission Deadline: January 29 2024
Paper Acceptance Notification: March 15, 2024
Final Paper Submission & Early Registration Deadline: May 1, 2024
Main Conference (IEEE WCCI 2024, Yokohama, Japan) : June 30 - July 5, 2024
Submission Format and Platform:
Submissions will be through the IEEE WCCI 2024 Submission page.
Each paper is limited to 8 pages, including figures, tables, and references. Please refer to the author guidelines provided by IEEE WCCI 2024.
Please specify during the submission that your paper is intended for the Special Session: Emergent Phenomena in Deep Representations and Large Language Models.
Contact information
Özge Alacam : oezge.alacam[at]uni-bielefeld.de
Michiel Straat : mstraat[at]techfak.uni-bielefeld.de