Sinong Wang

About Me

I am a Principal research Scientist in TBD Lab, Meta SuperIntelligence Labs. My research interests include large language model (LLM), natural language processing, machine learning and optimization. I used to be a graduate student (PhD) in the Department of Electrical and Computer Engineering, the Ohio State University, advised by Prof. Ness Shroff from 2015 September to 2019 April. I am interested in solving the challenging problems via non-trivial theoretical developments. I received the Outstanding paper award at NAACL 2024, Kenneth C. Sevcik Outstanding Paper Award at ACM SIGMETRICS 2017, and the best paper award at IEEE ICNC 2015.

Check out my Google Scholar if you are interested. 

My team is looking for 2025 summer research Intern. Please drop me an email if you are interested.

News

[Dec. 2021] Meta AI highlighted my team's latest effort in Our new AI system adapts to tackle quickly evolving harmful content. This is based on my Entailment as few-shot learner paper.

[Nov. 2021] Meta AI highlighted my team's latest effort in Using extremely generalizable AI to better identify violating content.

[Nov. 2020] Facebook AI highlighted my team's latest effort in How Facebook uses super-efficient AI models to detect hate speech. This is based on my Linformer paper.

[Nov. 2020] Facebook AI highlighted my team's latest effort in Training AI to detect hate speech in the real world. 

[May. 2020] Facebook AI highlighted my team's latest effort in AI advances to better detect hate speech.  

[Nov. 2019] Facebook CTO Mike Schroepfer highlighted how our team's AI techniques greatly helped Facebook detect harmful content and make Facebook a much safer place. More coverage at The Wall Street Journal and VentureBeat.

Publications

2025

Zishun Yu, Tengyu Xu, Di Jin, Karthik Abinav Sankararaman, Yun He, Wenxuan Zhou, Zhouhao Zeng, Eryk Helenowski, Chen Zhu, Sinong Wang, Hao Ma, Han Fang, Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization, ICML 2025

Yiqing Xie, Wenxuan Zhou, Pradyot Prakash, Di Jin, Yuning Mao, Quintin Fettes, Arya Talebzadeh, Sinong Wang, Han Fang, Carolyn Rose, Daniel Fried, Hejia Zhang, Improving Model Factuality with Fine-grained Critique-based Evaluator, ACL 2025.

Chengwei Qin, Wenxuan Zhou, Karthik Abinav Sankararaman, Nanshu Wang, Tengyu Xu, Alexander Radovic ~Alexander_Radovic1 , Eryk Helenowski, Arya Talebzadeh, Aditya Tayade, Sinong Wang, Shafiq Joty, Han Fang, Hao Ma, Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation, ACL 2025.

Chaoqi Wang, Zhuokai Zhao, Yibo Jiang, Zhaorun Chen, Chen Zhu, Yuxin Chen, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, Hao Ma, Sinong Wang, Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment, Arxiv.

Yen-Ting Lin, Di Jin, Tengyu Xu, Tianhao Wu, Sainbayar Sukhbaatar, Chen Zhu, Yun He, Yun-Nung Chen, Jason Weston, Yuandong Tian, Arash Rahnama, Sinong Wang, Hao Ma, Han Fang, Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback, Arxiv.

2024

Yun He, Di Jin, Chaoqi Wang, Chloe Bi, Karishma Mandyam, Hejia Zhang, Chen Zhu, Ning Li, Tengyu Xu, Hongjiang Lv, Shruti Bhosale, Chenguang Zhu, Karthik Abinav Sankararaman, Eryk Helenowski, Melanie Kambadur, Aditya Tayade, Hao Ma, Han Fang, Sinong Wang, Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following, Arxiv.

Yu Meng, Jitin Krishnan, Sinong Wang, Qifan Wang, Yuning Mao, Han Fang, Marjan Ghazvininejad, Jiawei Han, Luke Zettlemoyer, "Representation Deficiency in Masked Language Modeling", ICLR 2024.

Chaoqi Wang, Zhuokai Zhao, Chen Zhu, Karthik Abinav Sankararaman, Michal Valko, Xuefei Cao, Zhaorun Chen, Madian Khabsa, Yuxin Chen, Hao Ma, Sinong Wang, Preference Optimization with Multi-Sample Comparisons, Arxiv.

Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman, Di Jin, Kaiyan Peng, Eric Han, Shaoliang Nie, Chen Zhu, Hejia Zhang, Wenxuan Zhou, Zhouhao Zeng, Yun He, Karishma Mandyam, Arya Talabzadeh, Madian Khabsa, Gabriel Cohen, Yuandong Tian, Hao Ma, Sinong Wang, Han Fang, The Perfect Blend: Redefining RLHF with Mixture of Judges, Arxiv.

Contributor, The llama 3 herd of models, Meta AI.

Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang, Lm-infinite: Simple on-the-fly length generalization for large language models, Outstanding Paper Award, NAACL 2024.

Wenhan Xiong, Jinyu Liu, Igor Molybog, ... , Sinong Wang, Hao Ma, Effective long-context scaling of foundation models, NAACL 2024.

2023

Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Zenglin Xu, Shaoliang Nie, Sinong Wang, Madian Khabsa, Hamed Firooz and Dongfang Liu, "MUSTIE: Multimodal Structural Transformer for Web Information Extraction", ACL 2023.

Li Yang, Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Yu Chen, Madian Khabsa, Sinong Wang, Zenglin Xu and Dongfang Liu, "MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction", ACL 2023

Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang, Sinong Wang, Hamed Firooz, Hamed Pirsiavash, Liang Tan, Andreas Geiger, "Defending Against Patch-based Backdoor Attacks on Self-supervised Learning ", CVPR 2023.

2022

Qifan Wang, Li Yang, Jingang Wang, Jitin Krishnan, Bo Dai, Sinong Wang, Zenglin Xu, Madian Khabsa and Hao Ma, "SMUTPAVE: Structured Multimodal Transformer for Product Attribute Value Extraction", EMNLP 2022.

Qifan Wang, Li Yang, Xiaojun Quan, Fuli Feng, Dongfang Liu, Zenglin Xu, Sinong Wang and Hao Ma, "Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation", EMNLP 2022.

Yifang Chen, Karthik Sankararaman, Alessandro Lazaric, Matteo Pirotta, Dmytro Karamshuk, Qifan Wang, Karishma Mandyam, Sinong Wang, Han Fang, "Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler".

Karthik A Sankararaman, Sinong Wang, Han Fang, "BayesFormer: Transformer with Uncertainty Estimation".

Darsh Shah, Sinong Wang, Han Fang, Hao Ma, Luke Zettlemoyer, "Reducing Target Group Bias in Hate Speech Detectors".

Zhuofeng Wu, Sinong Wang, Jiatao Gu, Rui Hou, Yuxiao Dong, V.G.Vinod Vydiswaran, Hao Ma, "IDPG: An Instance-Dependent Prompt Generation Method", NAACL 2022.

Qinyuan Ye, Madian Khabsa, Sinong Wang, Xiang Ren, Mike Lewis, Aaron Jaech, "Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models", NAACL 2022.

Khalil Mrini, Shaoliang Nie, Jiatao Gu, Sinong Wang, Maziar Sanjabi, Hamed Firooz, "Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem", ACL 2022.

2021

Xuezhe Ma*, Xiang Kong*, Sinong Wang*, Chunting Zhou, Jonathan May, Hao Ma, Luke Zettlemoyer, "Luna: Linear Unified Nested Attention", NeurIPS 2021. [Code]

Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, Hao Ma, "Entailment as Few-Shot Learner".

Nayeon Lee, Belinda Z Li, Sinong Wang, Pascale Fung, Hao Ma, Wen-tau Yih, Madian Khabsa, "On Unifying Misinformation Detection", NAACL 2021.

Qinyuan Ye, Belinda Z Li, Sinong Wang, Benjamin Bolte, Hao Ma, Xiang Ren, Wen-tau Yih, Madian Khabsa, "On the Influence of Masking Policies in Intermediate Pre-training", EMNLP 2021.

2020

Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, Hao Ma, "CLEAR: Contrastive Learning for Sentence Representation".

Sinong Wang, Belinda Li, Madian Khabsa, Han Fang and Hao Ma, "Linformer: Self-Attention with Linear Complexity". [Code]

Sinong Wang, Madian Khabsa and Hao Ma, "To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks", ACL 2020.

Jiezhong Qiu, Hao Ma, Omer Levy, Wen-tau Yih, Sinong Wang and Jie Tang, "Blockwise Self-Attention for Long Document Understanding", EMNLP 2020.

Nayeon Lee, Belinda Li, Sinong Wang, Wen-tau Yih, Hao Ma, Madian Khabsa. "Language Models as Fact Checkers?", Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER), 2020.

2019

Sinong Wang, Jiashang Liu and Ness Shroff, "Fundamental Limits of Approximate Gradient Coding", SIGMETRICS 2020.

Sinong Wang, Jiashang Liu, Ness Shroff, and Pengyu Yang, "Computation Efficient Coded Linear Transform", AISTATS 2019.

2018

Sinong Wang, Jiashang Liu and Ness Shroff, "Coded Sparse Matrix Multiplication", ICML 2018.

Fang Liu, Sinong Wang, Swapna Buccapatnam, and Ness Shroff, "UCBoost: A Boosting Approach to Tame Complexity and Optimality for Stochastic Bandits", IJCAI 2018.

Sinong Wang and Ness Shroff. "Towards Fast-Convergence, Low-Delay and Low-Complexity Network Optimization", SIGMETRICS 2018.

2017

Sinong Wang and Ness Shroff, "A New Alternating Direction Method for Linear Programming", NIPS 2017. [Full version][Poster]

Sinong Wang and Ness Shroff, "Security Game with Non-additive Utilities and Multiple Attacker Resources",  Kenneth C. Sevcik Outstanding Paper Award, SIGMETRICS 2017. [SLIDES]

Sinong Wang, Fang Liu and Ness Shroff, "Non-additive Security Game", AAAI 2017 [Technical report] [SLIDES]

Before PhD

Sinong Wang, Xiaohua Tian and Hui Liu, "Exploiting the Unexploited of Coded Caching for Wireless Content Distribution", Best Paper Award, IEEE ICNC 2015. (selected 1 of 445)