Katie Luo

Katie Luo

Ph.D. Candidate @ Cornell University

Cornell University

About Me

Hello! I am a Ph.D. candidate at Cornell University, advised by Prof. Kilian Q. Weinberger and Prof. Bharath Hariharan. My research interest lies in visual understanding of the world, including 3D perception and multi-modal learning, combining visual data with other sensory inputs to enhance environmental understanding.

I am fortunate to have been supported by a Cornell University Fellowship, an Nvidia Graduate Student Fellowship, and an American Association of University Women (AAUW) Dissertation Fellowship Award. Before joining Cornell University, I had the pleasure of working on a variety of topics including applied machine learning, deep reinforcement learning, and applications of natural language processing. View my professional summary here.

📌 I will graduate in Spring 2025 and am actively seeking a postdoctoral or industrial position! Please reach out to me if you think I might be a good fit.

News:

[Oct. 2024] Presenting our work on bounding box refinement DiffuBox at NeurIPS 2024 in Vancouver, Canada.
[Oct. 2024] Presenting our work Denoising Vision Transformers at ECCV 2024 in Milan, Italy.
[May 2024] Received the AAUW American Dissertation Fellowship Award; notable alumnae include Marie Curie and Keisha Blain!
[Apr. 2024] Research exchange to the University of Sydney, Australian Centre for Robotics.
Interests
  • Machine Learning
  • Computer Vision
Education

  • PhD in Computer Science, 2020 - Present

    Cornell University

  • MS in Elec. Eng. and Computer Sciences, 2018 - 2019

    University of California, Berkeley

  • BSc in Elec. Eng. and Computer Sciences, 2015 - 2018

    University of California, Berkeley

Publications

(* denotes equal contribution)

For an up-to-date list of publications, check out my Google Scholar.

DiffuBox: Refining 3D Object Detection with Point Diffusion
NeurIPS 2024
3D bounding box refinement with the diffusion model object.
Xiangyu Chen*, Zhenzhen Liu*, Katie Z Luo*, Siddhartha Datta, Adhitya Polavaram, Yan Wang, Yurong You, Boyi Li, Marco Pavone, Wei-Lun Chao, Mark Campbell, Bharath Hariharan, Kilian Q. Weinberger
PDF Cite Code
DiffuBox: Refining 3D Object Detection with Point Diffusion
Denoising Vision Transformers
ECCV 2024  (Oral)
Investigating positional embedding artifacts in ViTs, and denosing them.
Jiawei Yang*, Katie Z Luo*, Jiefeng Li, Congyue Deng, Leonidas Guibas, Dilip Krishnan, Kilian Q. Weinberger, Yonglong Tian, Yue Wang
PDF Cite Code Project
Denoising Vision Transformers
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
ICRA 2024
The first work to investigate how to leverage standard definition maps to augment lane topology reasoning.
Katie Z Luo, Xinshuo Weng, Yan Wang, Shuang Wu, Jie Li, Kilian Q. Weinberger, Yue Wang, Marco Pavone
PDF Cite Code Project
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
Better Monocular 3D Detectors with LiDAR from the Past
ICRA 2024
Developed a method that uses repeated traversals to improve 3D object detection from camera inputs.
Yurong You*, Cheng Perng Phoo*, Carlos Andres Diaz, Katie Z Luo, Wei-Lun Chao, Mark Campbell, Bharath Hariharan, Kilian Q. Weinberger
Cite
Better Monocular 3D Detectors with LiDAR from the Past
Pre-training LiDAR-based 3D Object Detectors through Colorization
ICLR 2024
Self-supervised method of pre-training LiDAR based object detection leveraging camera color information.
Tai-Yu Pan, Chenyang Ma, Tianle Chen, Cheng Perng Phoo, Katie Z Luo, Yurong You, Mark Campbell, Kilian Q. Weinberger, Bharath Hariharan, Wei-Lun Chao
PDF Cite Code
Pre-training LiDAR-based 3D Object Detectors through Colorization
Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery
NeurIPS 2023
Discovering 3D objects with reward fine-tuning, drawing inspiration from the RL community.
Katie Z Luo*, Zhenzhen Liu*, Xiangyu Chen*, Yurong You, Cheng Perng Phoo, Sagie Benaim, Mark Campbell, Wen Sun, Bharath Hariharan, Kilian Q. Weinberger
PDF Cite Code
Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery
Unsupervised Adaptation from Repeated Traversals for Autonomous Driving
NeurIPS 2022
Domain adaptation method for 3D object detection leveraging repeated traversals.
Yurong You*, Katie Z Luo*, Cheng Perng Phoo*, Travis Zhang, Mark Campbell, Wei-Lun Chao, Bharath Hariharan, Kilian Q. Weinberger
PDF Cite Code
Unsupervised Adaptation from Repeated Traversals for Autonomous Driving
Learning to Detect Mobile Objects from LiDAR Scans Without Labels
CVPR 2022
Discovering 3D objects from repeated traversals and self training.
Yurong You*, Katie Z Luo*, Cheng Perng Phoo, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
PDF Cite Code
Learning to Detect Mobile Objects from LiDAR Scans Without Labels
Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features
BRAVO, ICCV 2023
Improving domain adaptation for 3D object detection with repeated traversals.
Travis Zhang, Katie Luo, Cheng Perng Phoo, Yurong You, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q Weinberger
PDF Cite Code
Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features
Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions
CVPR 2022
Introducing the Ithaca365 dataset, featuring diverse weather conditions and repeated traversals.
Carlos Andres Diaz, Youya Xia, Yurong You, Jose Nino, Junan Chen, Josephine Monica, Xiangyu Chen, Katie Z Luo, Yan Wang, Marc Emond, Wei-Lun Chao, Bharath Hariharan, Kilian Q. Weinberger, Mark Campbell
Cite
Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions
Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception
ICLR 2022
Leveraged repeated traversals to improve 3D object detection with LiDAR data.
Yurong You, Katie Z Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
PDF Cite Code Video
Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception
Residual Aligned: Gradient Optimization for Non-Negative Image Synthesis
CV4ARVR, ICCV 2021
Solving non-negative image generation at a local level.
Flora Yu Shen, Katie Luo, Guandao Yang, Harald Haraldsson, Serge Belongie
PDF Cite Code Video
Residual Aligned: Gradient Optimization for Non-Negative Image Synthesis
Stay Positive: Non-Negative Image Synthesis for Augmented Reality
CVPR 2021  (Oral)
Solving non-negative image generation for Augmented Reality.
Katie Luo*, Guandao Yang*, Wenqi Xian, Harald Haraldsson, Bharath Hariharan, Serge Belongie
PDF Cite Code Project Video
Stay Positive: Non-Negative Image Synthesis for Augmented Reality
Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
IROS 2021
Pedestrian motion forecasting with a focus on safety.
Katie Luo*, Sergio Casas*, Renjie Liao, Xinchen Yan, Yuwen Xiong, Wenyuan Zeng, Raquel Urtasun
PDF Cite Video
Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting
Implicit Latent Variable Model for Scene-Consistent Motion Forecasting
ECCV 2020
Motion forecasting algorithm using generative modeling for scene consistent predictions.
Sergio Casas*, Cole Gulino*, Simon Suo*, Katie Luo, Renjie Liao, Raquel Urtasun
PDF Cite Video
Implicit Latent Variable Model for Scene-Consistent Motion Forecasting
Learning robust rewards with adversarial inverse reinforcement learning
ICLR 2018
Introducing a practical adversarial inverse learning algorithm.
Justin Fu, Katie Luo, Sergey Levine
PDF Cite
Learning robust rewards with adversarial inverse reinforcement learning

Research

Academic Labs

 
 
 
 
 
Graduate Student Researcher
ML Core Lab at Cornell University, Advised by Prof. Kilian Q. Weinberger and Prof. Bharath Hariharan
Aug 2020 – Present Ithaca, USA
 
 
 
 
 
Visiting Student Researcher
Australian Centre for Robotics, Advised by Dr. Stephany Berrio Perez and Dr. Stewart Worrall
Mar 2024 – May 2024 Sydney, Australia
 
 
 
 
 
Visiting Student Researcher
Pioneer Center for AI, Advised by Prof. Serge Belongie
Jun 2022 – Aug 2022 Copenhagen, Denmark
 
 
 
 
 
AI Resident
Uber ATG (acquired by Aurora), Advised by Prof. Raquel Urtasun
Jun 2019 – Jun 2020 Toronto, Canada
 
 
 
 
 
Student Researcher
UC Berkeley Robotic and AI Learning Lab, Advised by Prof. Sergey Levine
Aug 2017 – Jun 2019 Berkeley, USA

Industry and Exchanges

A Couple of Featured Experiences

 
 
 
 
 
PhD Research Intern
Waymo – Perception Research
Jun 2024 – Oct 2024 Mountain View, USA
 
 
 
 
 
Research Intern
Nvidia – Autonomous Vehicle Research
Jun 2023 – Dec 2023 Santa Clara, USA
 
 
 
 
 
Research Intern
Facebook AI
May 2021 – Sep 2021 New York, USA
 
 
 
 
 
ML Engineering Intern
Google – YouTube
May 2018 – Aug 2018 Mountain View, USA
 
 
 
 
 
Data Science Intern
Wish, Context Logic
May 2017 – Aug 2017 San Francisco, USA
 
 
 
 
 
Software Engineering Intern
IBM
May 2016 – Aug 2016 San Jose, USA

Invited Talks

Leveraging Auxiliary Information for Self-Driving Perception across Diverse End Users
Colloquium talk for the Cornell University CS Ph.D. Students Colloquium. This talk discusses novel approaches to autonomous vehicle perception that reduce reliance on expensive 3D labeled datasets by leveraging auxiliary information sources like repeated traversal data and navigation maps.
Dec 5, 2024 Cornell University, New York
Unsupervised Objects Discovery from LiDAR Scans for Autonomous Driving
Seminar talk for the researchers from the Australian Centre for Robotics (ACFR) at the University of Sydney. This talk covered how we can shift away from supervised training methods for 3D detection in self-driving, and instead train 3D object detectors without any labels at all.
May 2, 2024 Australian Centre for Robotics, University of Sydney, Australia
Leveraging Auxiliary Information for Self-Driving Perception
The fast-forward talk following the Nvidia 2023 Graduate Student Fellowship year at the Nvidia GTC 2024. Discussed incorporating auxiliary information such as structure and standard definition maps to improve self-driving perception.
Mar 19, 2024 Nvidia GTC, San Jose Convention Center
The Under-Utilized Information in Repeated Traversals for Autonomous Vehicles
The inaugural talk for the P1 Cafe, Pioneer Center. Introducing works from the self-driving research group at Cornell University.
Jun 2, 2022 Pioneer Center, Copenhagen, Denmark