统计与数据科学系系列学术报告之四百零四期

时间：2023年7月25日（周二）15:00-16:00

主持人：复旦大学管理学院统计与数据科学系郁文教授

地点：李达三楼104室

报告人：Yang Feng Associate Professor ，School of Global Public Health，New York University

题目：Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness

摘要：Representation multi-task learning (MTL) and transfer learning (TL) have achieved tremendous success in practice. However, the theoretical understanding of these methods is still lacking. Most existing theoretical works focus on cases where all tasks share the same representation, and claim that MTL and TL almost always improve performance. However, as the number of tasks grows, assuming all tasks share the same representation is unrealistic. Also, this does not always match empirical findings, which suggest that a shared representation may not necessarily improve single-task or target-only learning performance. In this paper, we aim to understand how to learn from tasks with similar but not exactly the same linear representations, while dealing with outlier tasks. With a known intrinsic dimension, we propose two algorithms that are adaptive to the similarity structure and robust to outlier tasks under both MTL and TL settings. Our algorithms outperform single-task or target-only learning when representations across tasks are sufficiently similar and the fraction of outlier tasks is small. Furthermore, they always perform no worse than single-task learning or target-only learning, even when the representations are dissimilar. We provide information-theoretic lower bounds to show that our algorithms are nearly minimax optimal in a large regime. We also propose an algorithm to adapt to the unknown intrinsic dimension. We conduct two simulation studies to verify our theoretical results.

个人简介： Yang Feng's research interests encompass the theoretical and methodological aspects of machine learning, high-dimensional statistics, network models, and nonparametric statistics, leading to a wealth of practical applications. He has published over 60 peer-reviewed articles with over 3,600 Google Scholar Citations. He is currently an associate editor for the Annals of Applied Statistics, Journal of American Statistical Association, Journal of Business & Economic Statistics, and Statistica Sinica. His research has been supported by multiple grants from the National Science Foundation (NSF) and the National Institutes of Health (NIH). He is a fellow of the American Statistical Association (ASA), the Institute of Mathematical Statistics (IMS) and an elected member of the International Statistical Institute (ISI).