Knowledge Graph-Based Semantic System for

Visual Analytics in Automatic Manufacturing

Background

Visual analytics has been important for many data-driven applications in modern industries: exploratory data analysis for data understanding, visual anomaly detection, and results presentation of machine learning. Due to the importance and popularity, many tools and research have been proposed or conducted for visual analytics.

Challenges

However, how visual analytics are constructed is usually stored in the underlying code and suffers from transparency and re-usability.
Moreover, existing tools focus on the general purpose data visualisation, but limited exploited domain knowledge such as feature type information.

Our Solution: Overview

To this end, we propose a semantic system for user-friendly visual analytics, offering knowledge supported visual analytics with no-coding experience.

Our approach encodes visual analytical solutions in reusable knowledge graphs (referred to as visual KG) via graphic user interface (GUI) and reasoning, which can be translated to executable scripts [1].
Our approach incorporates domain knowledge in a type of automatic manufacturing, automated welding by linking domain ontologies and visual ontology (a set of axioms that encode domain knowledge of visual analytics, such as visualisation methods, procedure, and constraints) [2, 3].

System: (a) System for semantic visual analytics; (b) Template for Visual KG construction

Our Solution: System Description

From the bottom left, the Non-KG Data can be e.g., relational tables (csv), json files. For simplicity, we focus on relational tables. These data are mapped to the Domain Ontology via Data-to-Domain (Data2Domain) Mapping.

The Data2Domain Mapping maps tables and table columns to classes in the Domain Ontology .

Through Data Integration, domain KGs are generated. The Visual Ontology contains axioms that describe the knowledge for visual analytics.

The Visual KG Construction follows the schema defined by the Visual Ontology and generates Visual KG[5, 6], either manually[7] by a GUI or automatically following several Visual KG templates (also stored in the Visual Ontology ). The generated Visual KG can be stored in Visual KG Catalogue for later reuse.

The Visual KG Execution translates Visual KG to executable scripts and run the scripts, resulting the Visualisation Plots[8, 9].

Our Solution: Semantic Artefacts

The Data2Domain Mapping is provided by domain experts in the form of SQL queries on relational tables that return table columns corresponding to classes in the Domain Ontology.

We choose OWL 2 QL for the Domain Ontology because it is optimised for efficient query answering over relational databases.

We choose OWL 2 EL for the Visual Ontology because of its expressivity and it is still polynomial for query answering.

Between Data2Domain Mapping and the Domain Ontology there exist class links, since Data2Domain Mapping maps the Non-KG Data to classes in the Domain Ontology.

All classes in the Domain Ontology that represent features in data are linked to classes in the Visual Ontology via the subclass axioms.

Solving Visual Tasks with Reusable Visual KGs

The visual tasks in the use case can be categorised into two types:

I. Type I: Data inspection (Example 1), and Type I tasks are to inspect various data in exploratory data analysis for understanding the data.

II. Type II: Results visualisation (Example 2), and Type II tasks are to visualise results of statistical/ML analytics for discussing and interpreting the statistical/ML models for decision-making.

Example 1: (a) Example Visual KG; (b) the resulting plots for visualising ML prediction

Example 1 explanation:

Take the figure above as an example. Once the users choose to create a visual KG, the GUI will use the template in Figure of System b, which contains several owl:NamedIndividual of the types VisualPipeline, CanvasTask, PlotTask, and DescriptionTask. Next, the users will need to select the input data, and add several entities of PlotTasks from available tasks based on the Visual Ontology. For each PlotTask, the input data, the method and some parameters are mandatory to be given. The users need to configure the PlotTasks by specifying e.g., the inputs, line colour, line width. After that, they can also configure the CanvasTask and the DescriptionTask by giving the x label, y label, legend, etc.

Our approach first identifies there exists three input features with the labels: target, estimated training and estimated test. Then it generates a KG from the template in Figure of System b with three entities of PlotTasks between the CanvasTask and DescriptionTask. All of them are subclasses of TimeSeries. For TimeSeries, our approach uses the rule that adopts LinePlotMethod as the recommended plot method and adds its typing information:

Example 2: (a) Example Visual KG; (b) the resulting plots for inspecting the quality indicat

Example 2 explanation and Reusability:

For a new task of Type II and a new dataset in Example 2, the users can reuse the VisualPipeline in Example 1 a by simply modifying the input data entities (1), the plot methods (2), and properties of the plotting tasks (3, e.g., colour, marker size, labels, legend). This demonstrates the good reusability of our approach.

Industrial Scenario

Our evaluations are based on the following Industrial Use Case

We tested our approach in an industrial scenario of machine learning based quality monitoring for automated welding at Bosch.
We invited industrial users from Bosch, including non-data-scientists to workshops for evaluation.

Transparency and Coverage

Coverage
- After discussion, we categorised most tasks of visual encountered in our project in groups (as shown in Table), and give the coverage percentage according to our empirical cases. We can see that most of categories can be covered (above 80%).

Transparency
- We organised extensive workshops and collected 24 reports from Bosch welding experts, engineers, semantic experts, data scientists. They answered questions such as ''I found the semantic system helps to improve the transparency of visual analytics compared to the case without the approach'', and gave scores ranging 1-5 (from disagree, fairly agree, neural, fairly agree, to agree) which aggregated to 4.28 ± 0.47 (mean ± standard deviation) for the transparency.

Summary

We present our system of domain knowledge supported KG construction for visual analytics, which offers good reusablity, transparency and coverage for the visual analytic tasks.

We evaluated the system on a Bosch welding use case with promising results. As future work we plan to study hierarchical topic modelling to better organise our visual KG catalogue and push the deployment further. We also plan to further improve the system and organise demonstrations to a broader audience.

Authors and Affiliation

Authors: Baifan Zhou (1), Zhuoxun Zheng (2,3), Dongzhuoran Zhou (2,1), Zhipeng Tan (2,5),

Ognjen Savković (4), Hui Yang (6), Yujia Zhang (7) and Evgeny Kharlamov (2,1)

(1) SIRIUS Centre, Department of Informatics, University of Oslo, Norway

(2) Bosch Center for Artificial Intelligence, Germany

(3) Department of Computer Science, Oslo Metropolitan University, Norway

(4) Department of Computer Science, Free University of Bozen-Bolzano, Italy

(5) RWTH Aachen University, Germany

(6) Interdisciplinary Laboratory of Digital Sciences, France

(7) Department of Electrical and Computer Engineering , University of Alberta, Canada

Reference

[1] Dongzhuoran Zhou, Baifan Zhou, Zhuoxun Zheng, Zhipeng Tan, Egor V. Kostylev and Evgeny Kharlamov, Towards executable knowledge graph translation, in: ISWC, 2022.

[2] Zhuoxun Zheng, Baifan Zhou, Dongzhuoran Zhou, Gong Cheng, Ernesto Jimenez-Ruiz, Ahmet Soylu, and Evgeny Kharlamov, Query-based industrial analytics over knowledge graphs with ontology reshaping, ESWC, Springer (2022).

[3] Zhuoxun Zheng, Baifan Zhou, Ahmet Soylu and Evgeny Kharlamov, Towards a visualisation ontology for data analysis in industrial applications, in: SemIIM@ESWC, 2022.

[4] Baifan Zhou, Machine Learning Methods for Product Quality Monitoring in Electric Resistance Welding, Ph.D. thesis, Karlsruhe Institute of Technology, Germany, 2021.

[5] Dongzhuoran Zhou1,2(B) , Baifan Zhou2, Zhuoxun Zheng1,3, Egor V. Kostylev2, Gong Cheng4, Ernesto Jim´enez-Ruiz2,5, Ahmet Soylu3, and Evgeny Kharlamov1,2, Enhancing knowledge graph generation with ontology reshaping–Bosch case, ESWC, Springer (2022).

[6] Dongzhuoran Zhou1,2, Baifan Zhou2(B) , Zhuoxun Zheng1,3, Ahmet Soylu3, Gong Cheng4, Ernesto Jimenez-Ruiz2,5, Egor V. Kostylev2, and Evgeny Kharlamov1,2, Ontology reshaping for knowledge graph construction: Applied on bosch welding case, in: ISWC, 2022.

[7] Dongzhuoran Zhou, Baifan Zhou, Jieying Chen, Gong Cheng, Egor V. Kostylev, Evgeny Kharlamov, Towards ontology reshaping for KG generation with user-in-the-loop: Applied to bosch welding, in: IJCKG, 2022.

[8] Baifan Zhou, Zhuoxun Zheng, Dongzhuoran Zhou, Zhipeng Tan, Ognjen Savković, Hui Yang, Yujia Zhang and Evgeny Kharlamov, Knowledge graph-based semantic system for visual analytics in automatic manufacturing, in: ISWC, 2022.

[9] Baifan Zhou, Zhipeng Tan, Zhuoxun Zheng, Dongzhuoran Zhou, Ognjen Savkovic and Evgeny Kharlamov, Towards a visualisation ontology for reusable visual analytics, in: IJCKG, 2022.

Page updated

Google Sites

Report abuse