Research

Feature Learning and Representation Learning

Foundations of feature learning, with emphasis on multi-index, multi-scale, and compositional kernel learning.

Towards a Mathematical Theory of Feature Learning (PDF)

• A Compositional Kernel Model for Feature Learning
  Feng Ruan, Keli Liu and Michael Jordan
  Applied and Computational Harmonic Analysis, accepted, 2026.

• On the Kernel Learning Problem
  Yang Li and Feng Ruan
  Foundations of Computational Mathematics, accepted, 2026.

• Gradient Flow in the Kernel Learning Problem
  Yang Li and Feng Ruan.
  Preprint, 2025.

• A Theory of Feature Learning in Kernel Models
  Yunlu Chen, Yang Li, Keli Liu and Feng Ruan.
  Preprint, 2023.

• On the Self-Penalization Phenomenon in Feature Selection
  Michael Jordan, Keli Liu and Feng Ruan (Journal of Machine Learning Research, Major Revision).
  Under revision, 2023.

• A Self-Penalizing Objective Function for Scalable Interaction Detection
  Supplementary Material: [appendix.pdf]
  with Keli Liu.
  Arxiv (forever), 2020.

• LassoNet: A Neural Network with Feature Sparsity
  Ismael Lemhadri, Feng Ruan, Louis Abraham and Robert Tibshirani.
  Journal of Machine Learning Research, 22, 2021, 1-29.

Nonsmooth and Stochastic Optimization

Theory and algorithms for stochastic and nonsmooth optimization, including statistical and variational properties.

• On the Uniform Convergence of Subdifferentials in Stochastic Optimization and Learning
  Feng Ruan
  Mathematics of Operations Research, 2025.

• Asymptotic Optimality in Stochastic Optimization
  with John Duchi.
  The Annals of Statistics, 49, 2021, 21-48.

• Solving (most) of a set of quadratic equalities: Composite optimization for robust phase retrieval
  with John Duchi.
  Information and Inference, 8, 2019, 471-529.

• Stochastic Methods for Composite and Weakly Convex Optimization Problems
  with John Duchi.
  SIAM Journal on Optimization, 28, 2018, 3229–3259.

• Minimax Bounds on Stochastic Batched Convex Optimization
  with John Duchi and Chulhee Yun.
  Conference on Learning Theory (COLT), 2018.

• Sparse Recovery via Differential Inclusions
  with Stanley Osher, Jiechao Xiong, Yuan Yao and Wotao Yin.
  Applied and Computational Harmonic Analysis, 41, 2016, 436-469.

Other Work in Statistics and Machine Learning

Explorations of interesting problems I found worthwhile.

• The Generalization Error of Max-margin Linear Classifiers: High-dimensional Asymptotics in the Overparametrized Regime
  with Andrea Montanari, Youngtak Sohn and Jun Yan.
  The Annals of Statistics, 2025.

• Universality of max-margin classifiers
  Andrea Montanari, Feng Ruan, Basil Saeed and Youngtak Sohn.
  Preprint, 2023.

• A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
  with Xiang Li, Huiyuan Wang, Qi Long, Weijie J. Su
  The Annals of Statistics, 2025.

• Robust detection of watermarks for large language models under human edits
  with Xiang Li, Huiyuan Wang, Qi Long, Weijie J. Su
  Journal of the Royal Statistical Society Series B: Statistical Methodology, 2025.

• The Right Complexity Measure in Locally Private Estimation: It is not the Fisher Information
  with John Duchi.
  The Annals of Statistics, 2024.

• Is Temporal Difference Learning Optimal? An Instance-Dependent Analysis
  Koulik Khamaru, Ashwin Pananjady, Feng Ruan, Martin Wainwright, and Michael Jordan.
  SIAM Journal on Mathematics of Data Science, 3, 2022, 1013-1040.

• Multiclass Classification, Information, Divergence, and Surrogate Risk
  with John Duchi and Khashayar Khosravi.
  The Annals of Statistics, 46, 2018, 3246-3275.

• A Constrained Risk Inequality for General Losses
  with John Duchi.
  Artificial Intelligence and Statistics Conference (AISTATS), 2021.
  (oral presentation; top 10.5% of the accepted).

• Adapting to Unknown Noise Distribution in Matrix Denoising
  with Andrea Montanari and Jun Yan, 2018 (Bernoulli, Major Revision).
  Arxiv (forever), 2020.

• Characterizing Listener Engagement with Popular Songs Using Large-Scale Music Discovery Data
  Blair Kaneshiro, Feng Ruan, Casey W. Baker, and Jonathan Berge.
  Frontiers in Psychology, 8, 2017.

• A Tutorial on Libra: R package for the Linearized Bregman Algorithm in high dimensional statistics
  Jiechao Xiong, Feng Ruan and Yuan Yao.
  Springer Handbook on Big Data Analytics, 2017.