Ph.D in Information and Communication Technologies - ICT

Hyperdimensional Computing and Vector Symbolic Architectures for Artificial Intelligence

Academic Year 2024/2025

(12 hours, 3 credits)

Class Objectives

This course will present an emerging computing framework based on the use of high-dimensional vectors as a tool for representing and manipulating data structures in machine learning applications. This framework, commonly known as hyperdimensional computing (HDC) or symbolic vector architectures (VSAs), originated at the intersection of symbolic and connectionist approaches to artificial intelligence, but has morphed into an area of research in its own right. In recent years, there have been a growing number of applications in perception, analogical reasoning, memory models, and language processing. There are practical uses of this in different domains (ranging from neuroscience, computer science, electrical engineering, mathematics, and cognitive science) that are interesting also for advancing the study and interpretation of neural network models and their applications. The purpose of this course is to give Ph.D. students the tools to understand and manipulate HD vectors, employing them in different practical problems related to neural networks and artificial intelligence in general.

Prerequisites

Basic knowledge of linear algebra or abstract algebra, probability theory, elementary

logic; basic programming skills are not required but welcomed.

Class schedule 

All classes will be held in the DIET09 room (building code RM031)

Thursday 13th March 2025           16:00 - 18:30 

Thursday 20th March 2025           16:00 - 18:30 

Thursday 3rd April 2025              16:00 - 18:30 

Thursday 10th April 2025 16:00 - 18:30 

Syllabus

1.Introduction to Computing with High-dimensional Vectors

1.1 Overview of Traditional Computing

1.2 Introduction to Hyperdimensional Computing (HDC)

1.3 Vector Symbolic Architectures (VSA)

2. Overview of Different HDC/VSA Models

2.1 (Fourier) Holographic Reduced Representation

2.2 Binary Spatter Codes

2.3 Multiply Add Permute

2.4 Sparse Binary Distributed Representations

2.5 Other Models

3. Representation and Manipulation of Data Structures

3.1 Encoding and Decoding in HDC

3.2 Manipulating Data Structures in HDC

4. HDC/VSA and Artificial Intelligence

4.1 HD Vectors as Input/Output of (Randomized) Neural Networks

4.2 HDC/VSA in AI Model Design

4.3 Explainable AI HDC/VSA

5. Applications & Implementation

5.1 Cognitive Applications of HDC/VSA

5.2 Implementing HDC/VSA in Software and Hardware

5.3 Torch HD Library

5.3 Future Trends and Research Directions

Final Exam

Discussion of a scientific paper related to the course OR practical project on vertical

applications, inherent with students’ research programs and personal interests.

Course Material and bibliography 

Course slides, lecture notes, and code notebooks; available through the Google Classroom

page of the course (code fjes4vg).