# New Trends in

# Representation Learning with Knowledge Graphs

# Overview

Knowledge Graphs [Färber et al., 2018] are becoming the standard for storing, retrieving and querying structured data. In academia and in industry, they are increasingly used to provide background knowledge. Over the last years, several research contributions are made to show machine learning especially representation learning is successfully applied to knowledge graphs enabling inductive inference about facts with unknown truth values.

Several of these approaches [Nickel et al., 2011; Bordes et al., 2013; Wang et al., 2014] encode the graph structure that can be used for tasks such as link prediction, node classification, entity resolution, recommendation and many more. Although such graph representations can capture the complex relational patterns over multiple hops, they are still not enough to solve more complex tasks such as relational reasoning [Halford et al., 2010]. For this kind of tasks, we envision a need for representations with more expressive power, which could include representation in non-Euclidean space. This starts by capturing e.g., type constrained, transitive or hierarchical relations in an embedding [Xie et al., 15] up to learning expressive knowledge representations languages like first order logic rules.

Lately, the generalization of deep neural network models to non-Euclidean domains such as graphs and manifolds is explored [Bronstein et al., 2017]. They study the fundamental aspects that influence the underlying geometry of structured data for building graph representations [Hamilton et al., 2017; Nickel et al., 2018]. Recent advances in graph representation learning developed novel approaches such as convolutional neural networks for graphs [Defferrard et al., 2016, Niepert et al., 2016; Kipf et al., 2016; Chen et al., 2018], attention-based graph network [Veličković et al., 2018] etc. Most graphs here are either undirected or directed with both discrete and continuous node and edge attributes representing types of spatial or spectral data.

In this workshop we want to see how such novel representation learning methods can be applied to flexible relational reasoning tasks [Santoro et al., 2017; Garcia-Duran et al., 2017; Kipf et al., 2018] and what are its advantages in terms of expressive power, interpretability and generalization.

In this workshop we want to focus on the most exciting new developments in Knowledge Graph learning, bridging the gap to recent developments in different fields. Also, we want to bring together researchers from different disciplines but united by their adoption of earlier mentioned techniques from machine learning. We invite the submission of papers on topics including, but not limited to:

- Knowledge graph representations for relational reasoning
- Unsupervised learning of complex graphs over graph-structured data
- Neural/Statistical Relational Learning
- Integrating learning of expressive knowledge representation and flexible reasoning
- Exploring non-Euclidean spaces for knowledge graph representations
- Inference tasks for learned knowledge graph representations that require general-purpose reasoning
- Knowledge graph representations for industrial recommendation systems
- Decision modelling in personalized medicine with knowledge graph representations (e.g., decision support at the point of care in tumor boards)
- Visual scene graph modelling with the help of knowledge graphs.
- Knowledge graph representation to support natural language understanding.
- Knowledge Graphs for cognitive science
- Representation learning on time-dependent knowledge graphs
- Question answering and commonsense reasoning via knowledge graphs
- Knowledge graph representation learning models based on adversarial methods.
- Quantum Computing as a basis for scalable Knowledge graph representation learning.

**References**

- Färber, Michael, Frederic Bartscherer, Carsten Menne, and Achim Rettinger. "Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago."
*Semantic Web*9, no. 1 (2018): 77-129. - Nickel, Maximilian, Volker Tresp, and Hans-Peter Kriegel. "A Three-Way Model for Collective Learning on Multi-Relational Data." ICML. Vol. 11. 2011.
- Bordes, Antoine, et al. "Translating embeddings for modeling multi-relational data." Advances in neural information processing systems. 2013.
- Wang, Zhen, et al. "Knowledge graph embedding by translating on hyperplanes." Twenty-Eighth AAAI conference on artificial intelligence. 2014.
- Halford, Graeme S., William H. Wilson, and Steven Phillips. "Relational knowledge: The foundation of higher cognition." Trends in cognitive sciences 14.11 (2010): 497-505.
- Bronstein, Michael M., et al. "Geometric deep learning: going beyond euclidean data." IEEE Signal Processing Magazine 34.4 (2017): 18-42.
- Hamilton, William L., Rex Ying, and Jure Leskovec. "Representation learning on graphs: Methods and applications." arXiv preprint arXiv:1709.05584 (2017).
- Nickel, Maximilian, and Douwe Kiela. "Learning continuous hierarchies in the lorentz model of hyperbolic geometry." arXiv preprint arXiv:1806.03417 (2018).
- Niepert, Mathias, Mohamed Ahmed, and Konstantin Kutzkov. "Learning convolutional neural networks for graphs." International conference on machine learning. 2016.
- Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016).
- Chen, Jie, Tengfei Ma, and Cao Xiao. "Fastgcn: fast learning with graph convolutional networks via importance sampling." arXiv preprint arXiv:1801.10247 (2018).
- Veličković, Petar, et al. "Graph attention networks." arXiv preprint arXiv:1710.10903 (2017).
- Santoro, Adam, et al. "A simple neural network module for relational reasoning." Advances in neural information processing systems. 2017.
- Garcia-Duran, Alberto, and Mathias Niepert. "Kblrn: End-to-end learning of knowledge base representations with latent, relational, and numerical features." arXiv preprint arXiv:1709.04676 (2017).
- Kipf, Thomas, et al. "Neural relational inference for interacting systems." arXiv preprint arXiv:1802.04687 (2018).
- Xie, R., Liu, Z., & Sun, M. (2016, July). Representation Learning of Knowledge Graphs with Hierarchical Types. In IJCAI (pp. 2965-2971).
- Defferrard, M., Bresson, X., & Vandergheynst, P. (2016). Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems (pp. 3844-3852).