The future of large-scale experiments and their challenges in the digital era
Nature Human Behaviour, 2026 DOI 10.1038/s41562-026-02582-6
AI Research Scientist, Meta Super Intelligence Lab
I do model post-training for Meta AI and Muse, building agentic models aligned with what really delights the user.
Immediately before Meta, I was Head of Engineering, Azure Responsible AI, where I headed the engineering and science for Responsible AI in Microsoft’s Core AI division. Our platform safeguarded hundreds of billions of generative AI interactions annually, providing foundational safety systems for nearly all Azure OpenAI API traffic. This included multimodal content moderation, jailbreak prevention, cross-domain prompt injection protection (XPIA), hallucination detection, and protected material identification. We were also at the forefront of safety evaluation, red teaming, model steering, and establishing the trust and security foundations for agentic AI systems.
From 2020 to July 2024, I served as Technical Director of Platform Data Science and Machine Learning at Airbnb, where I chaired the science track of the company-wide AI & ML Council. My work spanned modernizing applied research workflows, developing a collaborative ML prototyping ecosystem, building foundational data infrastructure for event stream analytics and behavioral modeling, and advancing agile experimentation frameworks for offline and online model evaluation. I also held leadership roles in both the Relevance & Personalization team and the Core Data Science organization.
Earlier in my career, I was part of Microsoft’s Cloud & AI Platform, contributing to both the Microsoft Analysis and Experimentation Team and Azure Machine Learning. At the former, I led methodological and engineering improvements to one of the industry’s largest online experimentation platforms, with work spanning causal inference, Bayesian optimization, reinforcement learning, AutoML, recommender systems, high-throughput distributed systems, and analytics infrastructure. In Azure ML, I focused on the end-to-end experience for internal ML practitioners, including distributed training, automated ML pipelines, and product analytics.
I earned my Ph.D. in Statistics from Stanford University in 2010, where I studied sequential Monte Carlo methods under Prof. Tze Lai.
Nature Human Behaviour, 2026 DOI 10.1038/s41562-026-02582-6
arXiv 2026 · arXiv:2609.25408
arXiv 2026 · arXiv:2606.30992
KDD 2025
CIKM 2024
KDD 2024
The American Statistician, Vol. 78
arXiv 2024 · arXiv:2408.12748
Journal of Political Economy, 2020
AEA Papers and Proceedings, 2019
EC 2018
British Journal of Mathematical and Statistical Psychology, 2018 · psyarxiv
Statistics and Probability Letters, 2017
DSAA 2016 · ArXiv ver.
Statistics and Probability Letters, 2016
arXiv 2016 · arXiv:1610.03917
WWW 2015, The 1st Workshop on Offline and Online Evaluation of Web-based Services · Slides
WSDM 2015 · Slides
WSDM 2013
Operations Research, Feb 2012
Advances in Applied Probability, 2012
NABE Tech 2025
Work in progress
Amazon Tech Talk
Shaojie Deng — Interface 2011
Shaojie Deng, Roger Longbotham, Toby Walker and Ya Xu — JSM 2011