Sinong Wang

About Me

I am a Senior Staff research Scientist in Meta AI. My research interests include large language model (LLM), natural language processing, machine learning and optimization. I used to be a graduate student (PhD) in the Department of Electrical and Computer Engineering, the Ohio State University, advised by Prof. Ness Shroff from 2015 September to 2019 April. I am interested in solving the challenging problems via non-trivial theoretical developments. I received the Kenneth C. Sevcik Outstanding Paper Award at ACM SIGMETRICS 2017, and the best paper award at IEEE ICNC 2015.

Check out my Google Scholar if you are interested. 

I am looking for 2024 summer research Intern and AI resident. Please drop me an email if you are interested.

News

[Dec. 2021] Meta AI highlighted my team's latest effort in Our new AI system adapts to tackle quickly evolving harmful content. This is based on my Entailment as few-shot learner paper.

[Nov. 2021] Meta AI highlighted my team's latest effort in Using extremely generalizable AI to better identify violating content.

[Nov. 2020] Facebook AI highlighted my team's latest effort in How Facebook uses super-efficient AI models to detect hate speech. This is based on my Linformer paper.

[Nov. 2020] Facebook AI highlighted my team's latest effort in Training AI to detect hate speech in the real world. 

[May. 2020] Facebook AI highlighted my team's latest effort in AI advances to better detect hate speech.  

[Nov. 2019] Facebook CTO Mike Schroepfer highlighted how our team's AI techniques greatly helped Facebook detect harmful content and make Facebook a much safer place. More coverage at The Wall Street Journal and VentureBeat.

Publications

2023

Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Zenglin Xu, Shaoliang Nie, Sinong Wang, Madian Khabsa, Hamed Firooz and Dongfang Liu, "MUSTIE: Multimodal Structural Transformer for Web Information Extraction", ACL 2023.

Li Yang, Qifan Wang, Jingang Wang, Xiaojun Quan, Fuli Feng, Yu Chen, Madian Khabsa, Sinong Wang, Zenglin Xu and Dongfang Liu, "MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction", ACL 2023

Yu Meng, Jitin Krishnan, Sinong Wang, Qifan Wang, Yuning Mao, Han Fang, Marjan Ghazvininejad, Jiawei Han, Luke Zettlemoyer, "Representation Deficiency in Masked Language Modeling".

Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang, Sinong Wang, Hamed Firooz, Hamed Pirsiavash, Liang Tan, Andreas Geiger, "Defending Against Patch-based Backdoor Attacks on Self-supervised Learning ", CVPR 2023.

2022

Qifan Wang, Li Yang, Jingang Wang, Jitin Krishnan, Bo Dai, Sinong Wang, Zenglin Xu, Madian Khabsa and Hao Ma, "SMUTPAVE: Structured Multimodal Transformer for Product Attribute Value Extraction", EMNLP 2022.

Qifan Wang, Li Yang, Xiaojun Quan, Fuli Feng, Dongfang Liu, Zenglin Xu, Sinong Wang and Hao Ma, "Learning to Generate Question by Asking Question: A Primal-Dual Approach with Uncommon Word Generation", EMNLP 2022.

Yifang Chen, Karthik Sankararaman, Alessandro Lazaric, Matteo Pirotta, Dmytro Karamshuk, Qifan Wang, Karishma Mandyam, Sinong Wang, Han Fang, "Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler".

Karthik A Sankararaman, Sinong Wang, Han Fang, "BayesFormer: Transformer with Uncertainty Estimation".

Darsh Shah, Sinong Wang, Han Fang, Hao Ma, Luke Zettlemoyer, "Reducing Target Group Bias in Hate Speech Detectors".

Zhuofeng Wu, Sinong Wang, Jiatao Gu, Rui Hou, Yuxiao Dong, V.G.Vinod Vydiswaran, Hao Ma, "IDPG: An Instance-Dependent Prompt Generation Method", NAACL 2022.

Qinyuan Ye, Madian Khabsa, Sinong Wang, Xiang Ren, Mike Lewis, Aaron Jaech, "Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models", NAACL 2022.

Khalil Mrini, Shaoliang Nie, Jiatao Gu, Sinong Wang, Maziar Sanjabi, Hamed Firooz, "Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem", ACL 2022.

2021

Xuezhe Ma*, Xiang Kong*, Sinong Wang*, Chunting Zhou, Jonathan May, Hao Ma, Luke Zettlemoyer, "Luna: Linear Unified Nested Attention", NeurIPS 2021. [Code]

Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, Hao Ma, "Entailment as Few-Shot Learner".

Nayeon Lee, Belinda Z Li, Sinong Wang, Pascale Fung, Hao Ma, Wen-tau Yih, Madian Khabsa, "On Unifying Misinformation Detection", NAACL 2021.

Qinyuan Ye, Belinda Z Li, Sinong Wang, Benjamin Bolte, Hao Ma, Xiang Ren, Wen-tau Yih, Madian Khabsa, "On the Influence of Masking Policies in Intermediate Pre-training", EMNLP 2021.

2020

Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, Hao Ma, "CLEAR: Contrastive Learning for Sentence Representation".

Sinong Wang, Belinda Li, Madian Khabsa, Han Fang and Hao Ma, "Linformer: Self-Attention with Linear Complexity". [Code]

Sinong Wang, Madian Khabsa and Hao Ma, "To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks", ACL 2020.

Jiezhong Qiu, Hao Ma, Omer Levy, Wen-tau Yih, Sinong Wang and Jie Tang, "Blockwise Self-Attention for Long Document Understanding", EMNLP 2020.

Nayeon Lee, Belinda Li, Sinong Wang, Wen-tau Yih, Hao Ma, Madian Khabsa. "Language Models as Fact Checkers?", Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER), 2020.

2019

Sinong Wang, Jiashang Liu and Ness Shroff, "Fundamental Limits of Approximate Gradient Coding", SIGMETRICS 2020.

Sinong Wang, Jiashang Liu, Ness Shroff, and Pengyu Yang, "Computation Efficient Coded Linear Transform", AISTATS 2019.

2018

Sinong Wang, Jiashang Liu and Ness Shroff, "Coded Sparse Matrix Multiplication", ICML 2018.

Fang Liu, Sinong Wang, Swapna Buccapatnam, and Ness Shroff, "UCBoost: A Boosting Approach to Tame Complexity and Optimality for Stochastic Bandits", IJCAI 2018.

Sinong Wang and Ness Shroff. "Towards Fast-Convergence, Low-Delay and Low-Complexity Network Optimization", SIGMETRICS 2018.

2017

Sinong Wang and Ness Shroff, "A New Alternating Direction Method for Linear Programming", NIPS 2017. [Full version][Poster]

Sinong Wang and Ness Shroff, "Security Game with Non-additive Utilities and Multiple Attacker Resources",  Kenneth C. Sevcik Outstanding Paper Award, SIGMETRICS 2017. [SLIDES]

Sinong Wang, Fang Liu and Ness Shroff, "Non-additive Security Game", AAAI 2017 [Technical report] [SLIDES]

Before PhD

Sinong Wang, Xiaohua Tian and Hui Liu, "Exploiting the Unexploited of Coded Caching for Wireless Content Distribution", Best Paper Award, IEEE ICNC 2015. (selected 1 of 445)