统计与数据科学系系列学术报告之五百期

时间：2026年1月6日（星期二）16:00-17:00

主持人：复旦大学管理学院统计与数据科学系蒋斐宇教授

地点：史带楼303室

报告人：Dr. Kaizheng Wang

Columbia University

题目：Quantifying Fidelity in AI Persona Simulations

摘要：Large language models (LLMs) are increasingly used to simulate human behaviors such as public opinions and consumer choices. Yet, the unknown distribution shift between AI personas and the real populations they aim to emulate undermines the validity of conclusions. To tackle this challenge, we introduce an uncertainty quantification approach that converts imperfect simulated responses into confidence sets for population parameters of human responses. A key innovation lies in determining the optimal number of simulated responses: too many produce overly narrow confidence sets with poor coverage, while too few yield excessively loose estimates. Our method adaptively selects the simulation sample size and ensures valid average-case coverage guarantees. It is broadly applicable to any LLM, irrespective of its fidelity, and any procedure for constructing confidence sets. Crucially, the selected sample size also serves as an interpretable fidelity metric that directly quantifies the simulator-population misalignment. We illustrate this method on real-world datasets and LLMs.

个人简介：Kaizheng Wang is an assistant professor of Industrial Engineering and Operations Research, and a member of Data Science Institute at Columbia University. He works at the intersection of statistics, machine learning, and optimization. He received the ICBS Frontiers of Science Award in Mathematics in 2024, SIAM Activity Group on Imaging Science Best Paper Prize in 2024, and the Second Place Award in the 2023 INFORMS Data Mining Challenge.