INCITE Project  

Our ultimate goal is to contribute to advancing AI from narrow to “broad” (general) while ensuring AI Safety and alignment with human values, and contribute towards advances in other fields (healthcare, biomedical sciences, and others) via developing generic, powerful large-scale models pretrained in a self-supervised manner on broad variety of datasets. Such models can serve as a foundation of transferable knowledge and can be used in a broad variety of applications ("downstream tasks") due to their drastically improved generalization abilities as compared to prior state-of-art in the field of AI. 

It is observed that these "foundation models" appear to improve their generalization and few-shot learning abilities with scale, following certain empirical scaling laws. We plan to investigate this trend in more detail, and identify the most promising approaches to scaling the architecture, and datasets. Next, we plan to extend these model to handle a much wider range of modalities beyond text and images, as well as various machine-learning tasks, and expand them towards adaptive, continually learning systems. Finally, we plan to use the obtained foundation models for predictive modeling in several applications such as healthcare and brain imaging.  We aim towards highly transferable multi-modal models that we will make publicly available.  

Hence, we see this project as a step towards the democratization of large multi-modal models for the broader research community. The public availability of these models will allow researchers to investigate both strengths and weaknesses of large multi-modal models, and further improve their transfer capabilities to be used widely across various scientific domains.

Related presentation:
Towards General and Robust AI at Scale