Xin Zhong - Research Paradigm

My research investigates how to learn representations that simultaneously capture semantic content and remain invariant to distortions, enabling robust visual and multimodal intelligence and supporting applications such as watermarking and model attribution.

Worldview

My view is that AI models encode meaning and understand the learned world through Invariant Features.

These invariant features can be interpreted as semantically stable directions embedded in the local representation manifold geometry of the model. Around each local point, the model forms a local tangent-like semantic space, where infinitesimal movement along some directions preserves meaning, while movement along other directions changes the underlying concept.

For language models, invariant directions may correspond to semantic-preserving transformations such as paraphrasing, syntactic variation, or stylistic rewriting, while nuisance directions correspond to changes in the actual meaning or intent. [Ref: https://arxiv.org/abs/2605.06458, where we characterize the local geometry of language model representations and observe structured low-rank invariant subspaces, suggesting that language models capture semantic cores that remain stable under surface-level changes.]
For vision models, invariant directions may correspond to semantic-preserving distortions such as rotation, blur, illumination variation, compression, or viewpoint change, while nuisance directions correspond to changes in object identity, scene composition, or semantic content. (Unlike text, image representations contain substantially richer local variability due to fine-grained spatial, photometric, and structural details. As a result, visual invariant feature spaces are likely higher-dimensional and geometrically more complex than those in language, where semantic abstraction naturally compresses many variations into lower-rank structures.)

Importantly, to further the "undertsanding", Invariant Features from multimedia should remain aligned both locally and globally. [https://arxiv.org/abs/2602.18863; https://arxiv.org/abs/2503.13805]

Toward a Geometry of Intelligence (Ongoing)

If invariant features represent local principle constraints underlying learned notions, then intelligence can be interpreted as navigation over an information geometry defined by these invariant structures. In this view, the learned representation space forms a sparse semantic manifold field, where invariant directions define locally allowed semantic transport. Generalization may emerge as spontaneous transport toward structurally aligned invariant regions, even without explicit training on the target concept.

In language models (as illustrated in the figure), a prompt defines an initial point and initial semantic direction on this manifold. Reasoning can then be viewed as constrained transport through the densest invariant directions, where each generated token incrementally follows locally stable semantic pathways.
In vision models, the semantic manifold field defined by invariant features is likely substantially richer and more entangled than in language. A local image input corresponds not only to a semantic concept, but also to a dense configuration of spatial structure, texture, geometry, illumination, viewpoint, and object relationships. Consequently, transport on the invariant geometry does not simply correspond to sequential progression, but to continuous movement across locally allowed semantic-preserving directions in a high-dimensional visual manifold. In this view, visual understanding may emerge from constrained transport over sparse invariant structures that preserve semantic identity under changes such as rotation, blur, lighting variation, deformation, and viewpoint shift.