The Graph Module is the core of our solution, so we begin by illustrating its convergence process during the first cycle. We present results considering different initializations for node labels (random, uniform, gaussian or ground truth), experimentally proving that irrespective of the initialization, the method converges towards the same solution entirely defined by the Feature-Motion matrix (quantitative results of this experiment are presented in Fig. 6 of our original manuscript). This property of being able to converge to a stable solution, which naturally captures the main object in the scene, regardless of the initialization, is a key strength of our approach. It is entirely due to our unique Feature-Motion matrix formulation, which elegantly couples appearance and motion, offering the main object as its principal eigenvector in the space-time graph.

We present qualitative results of the evolution over cycles of both Graph Module and Network Module in the unsupervised setup (no pretraining using humanly annotated data). The given examples highlight the complementarity of the two modules and their agreement at convergence.  Both modules help each other from one iteration to the next.  Once again, we believe our method has two main advantages over previous ones: 1) the mathematical formulation and solution in the Graph Module is stable, converges, and does not depend on initialization (given a certain Feature-Motion matrix); the object is naturally discovered as the principal eigenvector of the Feature-Motion matrix. 2) The Feature-Motion matrix depends on the quality of features; we provide a way for learning such powerful features, in a self-supervised manner, by using the Net module. Therefore, the novelty of the Graph-Net dual system is in its ability to learn by itself more powerful features, which are then used by the Graph at the next cycle.

To highlight the power of our formulation, we provide a comparative analysis between our unsupervised IKE and IKE using pretrained features from an FCN backbone. We also present the segmentation maps generated by the FCN backbone to illustrate their limitations and highlight the advantages of our solution. Our self-supervised learned features follow the natural cluster can overcome heavily pretrained features.

• in the first subsequence containing an unusual bird (the Vogelkop superb bird-of-paradise) and in the subsequence with a parachute jumper, we observe how the supervised backbone is unable to detect the object, while our solution, both unsupervised and supervised, successfully handles the situation

• in sequences containing multiple objects, like the subsequence with breakdance or the subsequence with a camel, the supervised backbone extracts the masks of all the people/camels. With this prior, our supervised formulation is unable to delete the additional objects. We highlight that the unsupervised formulation works well in these scenarios and is not distracted by other objects

Qualitative comparison of our unsupervised solution with two state-of-the-art unsupervised methods (ELM and FST) highlighting our solution's spatio-temporal cluster consistency. 

• our soft-segmentation masks are cleaner, containing fewer background pixels than other solutions, resulting from the exploitation of the spatio-temporal consistencies (e.g., parkour and horseman subsequences)

• the segmentations of our IKE are more smooth and complete as a result of using the deep representation that is more locally consistent

Qualitative comparison of our unsupervised solution with two state-of-the-art supervised methods (3DC-Seg and MATNet) highlighting our solution's spatio-temporal consistency. In the supervised formulation, IKE receives the features of 3DC-Seg thus, we can observe its power with respect to the considered baseline.

• we observe how our solution can clean up some artifacts that are otherwise present in the 3DC-Seg (e.g., scooter subsequence)

• similar to the unsupervised setup, IKE results are generally smoother and containing fewer background pixels

The development of our system starts from three main assumptions regarding the object of interest: 1) has a different motion pattern than the background scene; 2) has a different appearance, statistically, than the scene; 3) is the main element of the scene in the sense that it indeed forms the strongest space-time cluster in the scene. When these assumptions do not hold in practice, our system fails to correctly identify the object considered by human annotators as the object of interest. In this video, we illustrate such scenarios.

• A. IKE assumes the existence of the main object, forming the main cluster of the Feature-Motion matrix. We illustrate scenarios when IKE focuses on multiple objects that share common motion and appearance patterns but are not all considered as main objects by the human annotators (e.g., the canal or the robot subsequences). We do not consider this as a true failure case, as our solution naturally finds those objects as the main cluster of the Feature-Motion matrix, although multiple strongly connected objects form this cluster. In cluttered, inconclusive scenes, the object selected is not necessarily the one indicated by the annotators (e.g., the carnival subsequence)

• B. IKE assumes that the main object has a distinctive appearance and motion pattern. If this is not the case, it will be unable to identify the main object. In the provided examples, the main objects are similar in both motion and appearance with the background area. In both subsequences, the motion is induced by camera motion.

• C. In IKE, the main object is the main cluster of the space-time graph, the one that stands out. In the case of static sequences, the motion contrast between object and background is missing (e.g., cow subsequence, where the object is almost static and far away from the camera). Also, there is not sufficient temporal change in the case of very short sequences to have a correct spatio-temporal clustering/separation from the background (e.g., subsequences with boat and cat)