We introduce PersonaMem, a personalization benchmark that features scalable and persona-oriented multi-session user-LLM conversations, as well as fine-grained in-situ user query types designed to evaluate LLM capabilities in memorizing, tracking, and incorporating users’ dynamic profiles into personalized responses across diverse scenarios.

@article{jiang2025know,

  title={Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale},

  author={Jiang, Bowen and Hao, Zhuoqun and Cho, Young-Min and Li, Bryan and Yuan, Yuan and Chen, Sihao and Ungar, Lyle and Taylor, Camillo J and Roth, Dan},

  journal={arXiv preprint arXiv:2504.14225},

  year={2025}

}

We propose a Retrieval-Augmented Generation (RAG)-based multi-agent LLM system to support analysis and decision-making in the context of natural hazards and extreme weather events. As a proof of concept, we present WildfireGPT, a specialized system focused on wildfire scenarios. The architecture employs a user-centered, multi-agent design to deliver tailored risk insights across diverse stakeholder groups.

@article{xie2025rag,

  title={A rag-based multi-agent llm system for natural hazard resilience and adaptation},

  author={Xie, Yangxinyu and Jiang, Bowen and Mallick, Tanwi and Bergerson, Joshua David and Hutchison, John K and Verner, Duane R and Branham, Jordan and Alexander, M Ross and Ross, Robert B and Feng, Yan and others},

  journal={arXiv preprint arXiv:2504.17200},

  year={2025}

@article{xie2024wildfiregpt,

  title={WildfireGPT: Tailored Large Language Model for Wildfire Analysis},

  author={Xie, Yangxinyu and Jiang, Bowen and Mallick, Tanwi and Bergerson, Joshua David and Hutchison, John K and Verner, Duane R and Branham, Jordan and Alexander, M Ross and Ross, Robert B and Feng, Yan and Levy, Leslie-Anne and Taylor, Camillo J},

  journal={arXiv preprint arXiv:2402.07877},

  year={2024}

}

This work demonstrates that diffusion models can achieve font-controllable multilingual text

rendering using just raw images without font label annotations. By integrating a conditional diffusion model with a text segmentation model, the method captures font styles in pixel space in a self-supervised manner, allowing user-specified font customization without ground-truth labels. 

@article{jiang2025controltext,

  title={ControlText: Unlocking Controllable Fonts in Multilingual Text Rendering without Font Annotations},

  author={Jiang, Bowen and Yuan, Yuan and Bai, Xinyi and Hao, Zhuoqun and Yin, Alyson and Hu, Yaojie and Liao, Wenyu and Ungar, Lyle and Taylor, Camillo J},

  journal={arXiv preprint arXiv:2502.10999},

  year={2025}

}

Unlike reasoning that aims to draw conclusions from premises, rationality ensures that those conclusions are reliably consistent, have an orderability of preference, and are aligned with evidence from various sources and logical principles. This survey is the first to comprehensively explore the notion of rationality in language and multimodal agents, analyzing how designs in existing agents and agent systems contribute to advancing certain key axioms of rationality.

@article{jiang2024towards,

  title={Towards Rationality in Language and Multimodal Agents: A Survey},

  author={Jiang, Bowen and Xie, Yangxinyu and Wang, Xiaomeng and Yuan, Yuan and Hao, Zhuoqun and Bai, Xinyi and Su, Weijie J and Taylor, Camillo J and Mallick, Tanwi},

  journal={arXiv preprint arXiv:2406.00252},

  year={2024}

}

@article{jiang2024multi,

  title={Multi-Modal and Multi-Agent Systems Meet Rationality: A Survey},

  author={Jiang, Bowen and Xie, Yangxinyu and Wang, Xiaomeng and Su, Weijie J and Taylor, Camillo J and Mallick, Tanwi},

  journal={arXiv preprint arXiv:2406.00252},

  year={2024}

}

We introduce Vysics, a vision-and-physics framework for a robot to build an expressive geometry and dynamics model of a single rigid body, using a seconds-long RGBD video and the robot’s proprioception. It uses a vision-based tracking and reconstruction method, BundleSDF, to estimate the trajectory and the visible geometry from an RGBD video, and an odometry-based model learning method, Physics Learning Library (PLL), to infer the “physible” geometry from the trajectory through implicit contact dynamics optimization. 

@article{bianchini2025vysics,

  title={Vysics: Object Reconstruction Under Occlusion by Fusing Vision and Contact-Rich Physics},

  author={Bianchini, Bibit and Zhu, Minghan and Sun, Mengti and Jiang, Bowen and Taylor, Camillo J and Posa, Michael},

  journal={arXiv preprint arXiv:2504.18719},

  year={2025}

}

@article{sun2023instance,

  title={Instance-Agnostic Geometry and Contact Dynamics Learning},

  author={Sun, Mengti and Jiang, Bowen and Bianchini, Bibit and Taylor, Camillo Jose and Posa, Michael},

  journal={arXiv preprint arXiv:2309.05832},

  year={2023}

}

We develop plug-and-play modules that enhance state-of-the-art scene graph generation methods to new levels of performance. Our approach integrates LLMs to critique predictions and reduce common sense violations, alongside a Bayesian classification scheme that leverages a hierarchical structure in relations for improved performance 

@article{jiang2023enhancing,

 title={Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge},

 author={Jiang, Bowen and Zhuang, Zhijun and Taylor, Camillo Jose},

 journal={arXiv preprint arXiv:2311.12889},

 year={2023}

}

@article{jiang2023scene,

 title={Scene graph generation from hierarchical relationship reasoning},

 author={Jiang, Bowen and Taylor, Camillo J},

 journal={arXiv preprint arXiv:2303.06842},

 year={2023}

}

We explore the zero-shot capabilities of foundation models in Visual Question Answering in the open world, and proposed an adaptive multi-agent system to address their limitations in object detection and counting. Instead of fine-tuning foundation models for specific datasets, our approach uses specialized agents as tools. 

@article{jiang2024multi,

  title={Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering},

  author={Jiang, Bowen and Zhuang, Zhijun and Shivakumar, Shreyas S and Roth, Dan and Taylor, Camillo J},

  journal={arXiv preprint arXiv:2403.14783},

  year={2024}

}

@article{shen2022batch,

  title={Batch active learning from the perspective of sparse approximation},

  author={Shen, Maohao and Jiang, Bowen and Zhang, Jacky Yibo and Koyejo, Oluwasanmi},

  journal={arXiv preprint arXiv:2211.00246},

  year={2022}

}