Manuscripts
Tao Yu, Congzheng Song, Jianyu Wang, Mona Chitnis. "Momentum Approximation in Asynchronous Private Federated Learning", (Under Review)
Tao Yu, Yichi Zhang, Zhiru Zhang, Christopher De Sa. "FedHDC: Secure and Private Federated Hyperdimensional Computing", (To Appear)
Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov. "Salvaging Federated Learning by Local Adaptation", [Code].
Conferences
Tao Yu, Gaurav Gupta, Karthick Gopalswamy, Amith R Mamidala, Hao Zhou, Jeffrey Huynh, Youngsuk Park, Ron Diamant, Anoop Deoras, Luke Huan. "Collage: Light-Weight Low-Precision Strategy for LLM Training". In 41th International Conference on Machine Learning (ICML 2024).
Tao Yu*, Toni J.B. Liu*, Albert Tseng, Christopher De Sa. "Shadow Cones: Unveiling Partial Orders in Hyperbolic Space", [Code]. In 12th International Conference on Learning Representations (ICLR 2024)
Albert Tseng, Tao Yu, Toni J.B. Liu, Christopher De Sa. "Coneheads: Hierarchy Aware Attention", [Code]. In 37th Conference on Neural Information Processing Systems (NeurIPS 2023)
Tao Yu, Christopher De Sa. "Random Laplacian Features For Learning with Hyperbolic Space" [Arxiv version], [ICLR version], [Code]. In 11th International Conference on Learning Representations (ICLR 2023)
Tao Yu*, Yichi Zhang*, Zhiru Zhang, Christopher De Sa. "Understanding Hyperdimensional Computing for Parallel Single-Pass Learning", [Code]. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022)
Tao Yu*, Wentao Guo*, Jianan Canal Li*, Tiancheng Yuan*, Christopher De Sa. "MCTensor: A High-Precision Deep Learning Library with Multi-Component Floating-Point", [Code]. In 39th International Conference on Machine Learning (ICML 2022), Workshop on Hardware Aware Efficient Training (HAET 2022)
Tao Yu, Christopher De Sa. "Representing Hyperbolic Space Accurately using Multi-Component Floats". In 35th Conference on Neural Information Processing Systems (NeurIPS 2021)
Tao Yu, Christopher De Sa. "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models", Spotlight, [Compression Code, Learning Code, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Tao Yu*, Shengyuan Hu*, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger. "A New Defense Against Adversarial Images: Turning a Weakness into a Strength", [Code, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Felix Wu*, Tianyi Zhang*, Amauri Holanda de Souza Jr.*, Christopher Fifty, Tao Yu, Kilian Q. Weinberger. "Simplifying Graph Convolutional Networks", [Code]. In 36th International Conference on Machine Learning (ICML 2019).
Tao Yu, Huan long, John Hopcroft. "Curvature-based Comparison of Two Neural Networks". In 24th International Conference on Pattern Recognition (ICPR 2018).
It's my fortune to intern at Apple MLPT Privacy team working with experienced reserachers and engineers including Ulfar Erlingsson, Vojta Jina, Martin Pelikan, Omid Javidbakht and etc., to look into some topics on federated learning and ml privacy.
I am glad to join Computer Science Department at Cornell University in fall 2019 as a Ph.D. student, it's my great fortune to work closely with Prof. Chris De Sa and Vitaly Shmatikov on some cool stuff.
Happy to get the research intern opportunity in Cornell CS from Prof. Kilian Q. Weinberger, to work on defenses against adversarial examples and simplifying GCN for NLP tasks. I also work closely with Prof. Chris De Sa on developing numerically robust and accurate models for hyperbolic embeddings of graphs.
It's my great honor to major in Mathematics and Applied Mathematics (ZhiYuan honours programme) at Shanghai Jiao Tong University, where I am so lucky to work with Prof. John E. Hopcroft and Huan Long, we analyzed both theoretically and experimentally of the intrinsic dimension of the manifolds embedded in neural networks.
Hyperbolic Space is particularly interesting and promising in machine learning due to its non-Euclidean properties. For example, volume of a ball in the hyperbolic space increases exponentially w.r.t. the radius (polynomially in Euclidean space). This imitates a tree, where the number of nodes increases exponentially over the depth (with a fixed branch factor).
We propose HyLa, a completely different approach to using hyperbolic space in graph learning: HyLa maps once from a learned hyperbolic-space embedding to Euclidean space via the eigenfunctions of the Laplacian operator in the hyperbolic space. HyLa is inspired by the random Fourier feature methodology, which uses the eigenfunctions of the Laplacian in Euclidan space. HyLa shows significant improvements over (hyperbolic) GCNs on downstream tasks including node classification and text classification.
We also look into a critical issue of using hyperbolic space: the NaN problem. We proposed tiling-based models and multi-component floats models to solve the NaN problem both theoretically and empirically, currently we are working on a Library to use these techniques easily in ML.
Despite of the great success of Machine learning, there are also some concerns calling for attention, namely, the security and privacy concern. On the one hand, Machine Learning models are vulnerable to imperceptible adversarial perturbations, which alter the model's decision entirely, it's necessary and worthwhile to design robust and secure models for various applications. On the other hand, Machine Learning models also suffer from information leakage, attacks such as membership inference and model inversion are able to infer information of the dataset. Hence, it's important to measure the information leakage and design privacy-preserving models and algorithms. However, both aspects may degrade the model's performace. What's more, it's particularly interesting to ask whether there is a tradeoff between robustness and privacy, we are currently looking at these tradeoffs in detail.
Federated learning is proposed for collaborative Machine Learning without centralized training data. Users will be able to collaboratively learn a shared model while keeping all the data on device. Latest FL approaches use differential privacy or robust aggregation to ensure privacy and integrity of the federated model, however, we show that these approaches will destroy the accuracy of the federated model for many participants. Thus, we propose local adaptation of federated models, our evaluatation of different techniques demonstrate that all participants benefit from local adaptation.
Professional Activities
PC/Reviewer
ICML, NeurIPS, ICLR, AISTATS, MLSys, KDD, SDM.
Talks
NeurIPS 2019, "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models", Slides.
VSAONLINE 2022, "Understanding hyperdimensional computing for parallel single-pass learn-
ing", Video, Slides.
Workshop on
Privacy Preserving Machine Learning 2024, "Efficient Asynchronous Private Federated Learning".