OVERVIEW

• Transfer-based adaptive tree for multi-modal sentiment analysis based on user latent aspects

Abstract - Multimodal sentiment analysis aims to extract positive or negative opinions of individuals based on visual, textual, and acoustic input channels, benefiting multiple real-world applications, including human-computer interaction and recommendation systems. Although the practitioners theoretically affirm the association between cognitive cues and emotional manifestations, most of the current multimodal approaches in sentiment analysis disregard user-specific latent aspects. To tackle this issue, in this work, we devise a novel unified framework to perform multimodal sentiment prediction by leveraging the composer's cognitive cues, such as personality. The proposed framework evolves an adaptive tree with hierarchically dividing users and training submodels that transfer cognitive-oriented knowledge by a modified attention-based fusion in an LSTM-based architecture. Finally, a prediction module performs sentiment prediction by consuming conclusive agglomerative knowledge from the adaptive tree. To avoid data sparsity in submodel construction, we utilize a dynamic dropout approach in learning to borrow data from neighboring nodes. The empirical results on two real-world datasets determine that our proposed model for sentiment prediction can surpass trending rivals. Furthermore, as results confirm, compared to other ensemble approaches, the proposed transfer-based model can better utilize the latent cognitive cues and consequently foster the prediction results. Finally, given the extrinsic and intrinsic analysis outcomes, the proposed hierarchical clustering approach on the adaptive tree can group the users better than the other devised theoretical-based technique.

CODE

• For the codes and the dataset please contact the email at the bottom of the page

PUBLICATIONS

• Najafipour, Saeed, Saeid Hosseini, Wen Hua, Mohammad Reza Kangavari, and Xiaofang Zhou. "SoulMate: Short-text author linking through Multi-aspect temporal-textual embedding." IEEE Transactions on Knowledge and Data Engineering (2020).

• Hosseini, Saeid, Saeed Najafipour, Ngai-Man Cheung, Hongzhi Yin, Mohammad Reza Kangavari, and Xiaofang Zhou. "TEAGS: time-aware text embedding approach to generate subgraphs." Data Mining and Knowledge Discovery 34 (2020): 1136-1174.

• Hosseini, Saeid, Hongzhi Yin, Xiaofang Zhou, Shazia Sadiq, Mohammad Reza Kangavari, and Ngai-Man Cheung. "Leveraging multi-aspect time-related influence in location recommendation." World Wide Web (2017): 1-28.

• Hosseini, Saeid, Hongzhi Yin, Meihui Zhang, Yuval Elovici, and Xiaofang Zhou. "Mining Subgraphs From Propagation Networks Through Temporal Dynamic Analysis." In 2018 19th IEEE International Conference on Mobile Data Management (MDM), pp. 66-75. IEEE, 2018.

• Hosseini, Saeid, Hongzhi Yin, Ngai-Man Cheung, Kan Pak Leng, Yuval Elovici, and Xiaofang Zhou. "Exploiting Reshaping Subgraphs from Bilateral Propagation Graphs." In International Conference on Database Systems for Advanced Applications, pp. 342-351. Springer, Cham, 2018.

CONTACT US

Ms. Sana Rahmani, rahmany.sana [at] gmail [dot] com

Dr. Saeid Hosseini, ssaeidhosseini [at] gmail [dot] com