Visual Grounding is basically an object detection task. One of the leading methods for object detection is DETR (Detection Transformer). In DETR, we take the image features, usually obtained using CNN, a ResNet-50 in this case. A positional embedding is added to the image features and the resulting features is passed through a Transformer Encoder. The Transformer Decoder takes a fixed number of learned positional encodings called object queries along with the outputs of the encoder. The outputs of the decoder are then passed through a feed forward network to either detect and object (class+bounding box) or "no object" class.

TransVG is also a transformers based take on Visual grounding. Here, the there are initially 2 separate branches, the visual branch and the text branch. The features are first extracted using a CNN backbone and a Language model respectively. These features are then passed through a visual transformer and a linguistic transformer respectively. These outputs of the 2 branches are then linearly projected to a shared space and concatenated. This is then passed through a Visual Linguistic Transformer. A prediction head is then used to regress the box coordinates.