Upcoming role of topology preservation in future data science

Introduction

Laymen explanation

Topology is at present less explored in machine learning, which is also why it is important to make it more available to the machine learning community at large.

As an example, Topological information can be incorporated effectively to add regularisation or incorporate prior knowledge into problems. Regularisation is used throughout machine learning to prevent over-fitting, or to solve ill-posed problems.

If you are interested to know about this, then this document helps.

Technical explanation

We often use machine learning to try to uncover patterns in data. In order for those patterns to be useful they should be meaningful and express some underlying structure. Geometry deals with such structure, and in machine learning we especially leverage local geometry. This can be seen in the Euclidean-inspired loss functions we use for generative models as well as for regularization. However, global geometry, which is the focus of Topology, also deals with meaningful structure, the only difference being that the structure is global instead of local. Topology is at present less exploited in machine learning, which is also why it is important to make it more available to the machine learning community at large.

Look at above example. Left picture is before training with topology loss and Right picture is after training with topology loss

Jargons understanding

Topology

In mathematics, Topology is the study of geometric properties that are preserved under continuous deformation. Example of deformations are stretching, twisting, crumpling and bending. Note that tearing or gluing is not included.

Many of us have seen the continuous deformation of a mug into a donut used to explain topology.

Topology prior

Topology prior is used to add topological information to incorporate prior knowledge into problems. Below picture shows example of three-layer torodial structure as a topology prior (Refer here)

Role in machine learning

• In many situations, it is possible to establish an invertible map from the birth and death time of a topological feature to a pair of points in the data. This map allows us to backpropagate from a loss function on the persistence diagram (the list of topological features with their birth and death times) to the underlying data.

  • Topological information can be incorporated effectively to add regularization or incorporate prior knowledge into problems. Regularization is used throughout machine learning to prevent over-fitting, or to solve ill-posed problems.

• Topological priors can be used to improve the quality of a deep generative neural network. Specifically, we want to improve its topological fidelity and the right number of local maxima.

Current status in ML

This is a nascent field and there is lots of scope for innovation.

When it comes to machine learning, topology is not as ubiquitous as local geometry, but in almost all cases where local geometry is useful so is topology. Topology is at present less explored in machine learning, which is also why it is important to make it more available to the machine learning community at large. Here is one paper which uses topology preservation approach.

This another paper talks about its application in robotics.

Similarly an application in CNN is mentioned in this paper.

Reference

https://youtu.be/q8gVpKl1f-4?t=2487

http://ai.stanford.edu/blog/topologylayer/

https://en.wikipedia.org/wiki/Topology

https://papers.nips.cc/paper/2019/file/2d95666e2649fcfc6e3af75e09f5adb9-Paper.pdf

https://arxiv.org/abs/1908.08870

https://arxiv.org/abs/1908.08870