Seminar Schedule

Opening Session

Prof. NGUYEN, Minh Le and Prof. Qiang Ma

Weakly-supervised Graph Representation Learning

Speaker: Assistant Prof. Yijun Duan

Talk Abstract: Graph representation learning (GRL) is a rapidly growing field. It has attracted significant amounts of attention from different domains and consequently accumulated a large body of literature. However, a large fraction of GRL methods require massive resources, including data collection, label annotation, and computation resources, which may be insufficient in real-world applications. The constraints of resources limit the deployment of existing graph neural networks (GNNs) models, resulting in their degraded performance. It is then a very challenging task to bridge the gap between graph representation learning and imperfect resource-limited real-world environments and application scenarios. To this end, I am working in developing novel weakly-supervised GRL methods that reduce the requirement of resources while maintaining the performance on par with resource-rich models to the maximum extent. I will address this challenge from the following three aspects: imbalanced GRL, incomplete GRL and inexact GRL.

ZeFaV: Boosting Large Language Models for Zero-shot Fact Verification

Speaker: Nguyen Khac Vu Hiep

Abstract: In this paper, we propose ZeFaV - a zero-shot based fact-checking verification framework to enhance the performance on fact verification task of large language models by leveraging the in-context learning ability of large language models to extract the relations among the entities within a claim, re-organized the information from the evidence in a relationally logical form, and combine the above information with the original evidence to generate the context from which our fact-checking model provide verdicts for the input claims. We conducted empirical experiments to evaluate our approach on two multi-hop fact-checking datasets including HoVer and FEVEROUS, and achieved potential results results comparable to other state-of-the-art fact verification task methods.

A Method for Controlling Knowledge Levels of LLM Agents to Support Self-Directed Learning 

Speaker: Kosei Nakazono

Abstract: Large Language Models (LLMs) are being explored for self-directed learning support systems. However, directly employing LLM agents in learner interactions risks providing excessive information, potentially hindering learning effectiveness. This study proposes a method to regulate LLM agent output based on the learner's estimated knowledge level to provide the optimal amount of information. A knowledge graph representing the learning objectives is first established. The learner's knowledge level is then inferred from the agent's and the learner's dialogue history. This inferred knowledge level dynamically controls the activation and decay of information within the agent's knowledge graph. The LLM agent's responses are generated based on this controlled knowledge graph. By implementing agents with varying knowledge levels, simulating learners, friends, teaching assistants (TAs), and instructors, we aim to construct a comprehensive self-directed learning support system.

ANSPRE: Improving Question-Answering in Large Language Models with Answer-Prefix Generation 

Speaker: Le Nguyen Khang

Abstract: Large language models (LLMs) and Retrieval-Augmented-Generation (RAG) show remarkable capabilities in Open-domain question-answering (ODQA). Despite the advancements, LLMs tend to generate verbose responses, of which only a small part is the answer phrase. Although the ability to produce the confidence score for the answer is essential when deploying LLMs in high-risk domains, sequence probabilities obtained from LLMs do not correlate well with the probabilities of correctness and thus fail to represent confidence scores. This study introduces Answer-prefix Generation (Anspre) to improve generation quality, allowing the LLMs to output answer phrases and produce highly reliable confidence scores. We guide the model in predicting the answer phrase using an answer prefix and design a ranking score that integrates parametric and non-parametric knowledge. The answer phrases and their corresponding scores enable Anspre to aggregate results from different documents and samplings to boost performance and produce confidence scores highly correlated with correctness. We show that Anspre can be applied to any LLM and present an approach called Self-Anspre to combine Anspre with Self-reflective RAG, a state-of-the-art framework based on reflection tokens. Empirical evaluation on popular ODQA benchmarks shows that Anspre and Self-Anspre significantly improve state-of-the-art LLMs and RAG frameworks. An in-depth analysis shows that confidence scores produced by Anspre are highly correlated to the likelihood of correctness.

Edge Classification on Imbalanced Multi-relational Graphs 

Speaker: Zhaojie Gong

Abstract: On real-world multi-relational graphs, edge labels often follow an imbalanced long-tail distribution, posing significant challenges to edge representation learning and subsequent applications, such as edge classification and prediction. This arises because edge features tend to be biased toward majority edge features, resulting in inadequate learning of features for minority class edges. We propose a novel method for tackling edge representation learning on imbalanced multi-relational graphs to address this issue. The key technical contributions include a synthetic minority edge generation approach inspired by SMOTE, combined with the construction of connecting links to enhance message passing on the augmented graphs. Extensive experiments on diverse real-world datasets demonstrate the effectiveness of the proposed method compared to competitive baselines.

Enhancing Knowledge Integration of LLMs with Knowledge Graph 

Speaker: Nguyen Tan Minh

Abstract: Large language models (LLMs) have made a great contribution to many aspects of NLP. Despite their impressive accomplishments, LLMs, even with retrieval-augmented generation (RAG), still struggle to efficiently integrate a large number of new experiences after pre-training. This study introduces a new framework inspired by the long-term memory theory ofthe human brain to enable deeper and more effective knowledge integration over new experiences. The framework synergistically utilizes LLMs, knowledge graph, and Personalized PageRank algorithm to mimic the different functions in human memory. The proposed framework outperforms state-of-the-arts methods on multi-hop question answering (QA) benchmarks remarkably, by up to 20. This study can also tackle new types of scenarios, which would be promising to domain-specific fields, e.g. medicine, fintech, law. .

Leveraging Heterogeneous Text Data for Reinforcement Learning-based Stock Portfolio Management 

Speaker: Keishi Fukuda

Abstract: We are researching stock trading support systems that integrate heterogeneous text data. In our prior work, we have proposed a reinforcement learning-based method that combines texts impacting prices in the short-term, such as social media posts, with texts affecting stock prices in the medium- to long-term, like earning reports. We propose the portfolio concept to develop a trading support method focused on risk diversification. We propose a text embedding approach using FinGPT, a finance-specialized language model, to enhance text feature extraction.

Efficiently Updating Large Language Models’ Memory 

Speaker: Nguyen The Hai

Abstract: Large Language Models (LLMs) are well-known as the new assistants of humans because they can deeply understand and generate text that helps people with a wide range of tasks. However, these models are limited to correctly answering questions related to the latest news about the world, e.g., who is the current president of America? This constraint is known as “knowledge cutoff”, which hinders these LLMs’ performance to the world’s dynamic nature. To this end, recent years have seen rapid growth in the technique of knowledge editing for LLMs, which aims to keep the models’ knowledge up-to-date with the current active world efficiently and effectively. Updating the LLM’s memories efficiently is one of the key objectives of our recent research since it can enable the community to build their specialized language agents at an affordable cost..

Overview of Multimodal Fact-checking on News 

Speaker: Luu Thanh Son

Abstract: Automated Fact-Checking is crucial for minimizing the effect of misleading information in the vast development of social networks era. However, fact-checking is a challenging task since the claim can have multiple evidence and the information in the evidence is diverse and complex, which needs an efficient method to combine and process to exploit valuable insight for verification of the truthfulness of the claim. Hence, we proposed a fusion framework that combines the claim in text form with evidence in both image and text to solve the claim verification task and generate the ruling sentence for the input claim. As an important aspect, we conduct various experiments with different text and image pre-trained models to investigate their impact on extracting useful features. The experimental results show that under the fusion setting in multimodal, our framework is efficient for the Automated Fact-Checking task when evaluating the claim verification task with Mocheg and FACTIFY datasets.

Deep Reasoning of Large Language Model on Knowledge Graph 

Speaker: Vo Thien Trung

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities across various natural language processing tasks. However, they often encounter challenges such as hallucinations and a lack of explainability, particularly in scenarios requiring complex, multi-hop reasoning. To address these issues, integrating LLMs with external Knowledge Graphs (KGs) has been proposed. This integration enables LLMs to interactively explore related entities and relationships within KGs, thereby enhancing their reasoning capabilities. By leveraging the structured information in KGs, LLMs can achieve deeper reasoning, improved knowledge traceability, and greater flexibility without necessitating additional training. This approach not only bolsters the performance of smaller LLMs, making them competitive with larger models like GPT-4 in certain contexts, but also reduces deployment costs. We plan to propose our method to achieve state-of-the-art results in multiple datasets, underscoring its effectiveness in augmenting LLM reasoning through the incorporation of KGs.

Closing Session

Prof. NGUYEN, Minh Le and Prof. Qiang Ma