About Me
Hi! I’m a Ph.D. student in EECS at UC Berkeley, where I’m fortunate to be advised by Prof. Yi Ma and Prof. Jiantao Jiao. I’m affiliated with BAIR and supported by a UC Berkeley College of Engineering fellowship. Prior to my PhD, I completed a BA in CS and MS in EECS, also at UC Berkeley.
My research interests broadly lie in simplifying deep learning. More specifically, I’m interested in developing theory to understand, improve, and simplify empirical deep learning methodology. I work on this problem through the following research threads:
- understanding the latent representations and internal algorithms learned by modern deep neural networks, usually from the perspectives of signal processing, statistics, and information theory;
- leveraging such-obtained conceptual insights to design interpretable, efficient, and principled improvements and alternatives to existing deep learning practices and methods.
I’m particularly interested in problem instances where data is high-dimensional yet has rich structure, such as computer vision and natural language processing.
In my free time, I play basketball, chess, and TFT, and read sci-fi novels.
Notes for undergraduate and masters students.
Note 1: I'm happy to chat about my research or general advising. Please send me an email and we can work out a time.
Note 2: If you are interested in research collaboration, please send me an email with your background and specific interests (the more detailed, the better). The recommended time investment is at least 15 hours per week. Unfortunately, right now my schedule is tight and generally does not permit consistent long-term mentoring of younger students, so some degree of self-sufficiency would be highly valued. To ensure a more fruitful collaboration, it would be best to have the technical knowledge to read and understand deep learning papers, especially theory-oriented work. Thank you for your understanding.
Recent Updates
- (May 2024) Our new comprehensive paper White-Box Transformers via Sparse Rate Reduction: Compression is all There Is?, reviewing our “White-Box Transformers” line of work: deriving efficient, interpretable, and performant transformer-like architectures from first-principles information theory and signal processing, was accepted to JMLR.
- (May 2024) Our paper A Global Geometric Analysis of Maximal Coding Rate Reduction was accepted to ICML 2024.
- (January 2024) Our paper Masked Completion via Structured Diffusion with White-Box Transformers, which develops a connection between iterative denoising in diffusion models and representation learning in transformer-like deep networks, and uses it to construct a performant, efficient, and interpretable transformer-like autoencoder, was accepted to ICLR 2024.
- (November 2023) Our papers Emergence of Segmentation with Minimalistic White-Box Transformers, Closed-Loop Transcription via Convolutional Sparse Coding, and Masked Completion via Structured Diffusion with White-Box Transformers were accepted to CPAL 2024.
- (October 2023) Our paper Emergence of Segmentation with Minimalistic White-Box Transformers was accepted to NeurIPS 2023 XAIA Workshop.
- (September 2023) Our paper White-Box Transformers via Sparse Rate Reduction, proposing an interpretable and parameter-efficient transformer-like architecture derived from first-principles, was accepted to NeurIPS 2023.
- (August 2023) Started my Ph.D. program in EECS at UC Berkeley!