👨‍🎓 About Me

Hi! I am currently a third-year PhD student at Institude for AI Industry Research (AIR) and School of Vehicle and Mobility, Tsinghua University, advised by Prof. Xianyuan Zhan and Prof. Ya-Qin Zhang. I got my bachelor’s degree in June 2021 from the School of Mechanical Engineering, Xi’an Jiaotong University.

My research interest broadly lies in advanced data-driven learning theory and algorithms on decision making and optimization, such as offline reinforcement learning (RL), as well as their promising applications on autonomous driving and robotics.

🔥 News

• [2024.05] DecisionNCE has been accepted in ICML 2024!
• [2024.02] Happy to announce our work on multimodal representations for robotics🤖, leveraging internet-scale video/language to assist decision making: DecisionNCE
• [2024.01] One paper “Query-Policy Misalignment in Preference-Based Reinforcement Learning” has been selected as Spotlight in ICLR 2024!
• [2024.01] Two papers “Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model” and “Query-Policy Misalignment in Preference-Based Reinforcement Learning” have been accepted to ICLR 2024!
• [2023.11] Honored to be selected as a Top Reviewer of NeurIPS 2023 (top 10%). Check here for details.
• [2023.03] One paper “Query-Policy Misalignment in Preference-Based Reinforcement Learning” has been accepted in ICML 2023 MFPL Workshps!
• [2023.02] Our three recent papers “Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization”, “When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning” and “Mind the Gap: Offline Policy Optimization for Imperfect Rewards” have been accepted in ICLR 2023!
• [2022.09] One paper “A Policy-Guided Imitation Approach for Offline Reinforcement Learning” has been accepted in NeurIPS 2022!

📝 Publications

* Equal contribution.

Preprints and Codebases

• Jianxiong Li$^*$, Shichao Lin$^*$, Tianyu Shi, Chujie Tian, Yu Mei, Jian Song, Xianyuan Zhan, Ruimin Li, A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning Under Review, 2023.

• Xiangyu Zhu$^*$, Jianxiong Li$^*$, Haoyi Niu$^*$, Yinan Zheng$^*$, Peng Cheng$^*$, Wenjia Zhang$^*$, Haoran Xu$^*$ and Xianyuan Zhan, D2C: A Data-Driven Control Library Based on Reinforcement Learning, Paper coming soon, 2023. [Code][Documentation]

• Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang, PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning Under Review, 2023. [Code]

• Haoran Xu, Xianyuan Zhan, Jianxiong Li, Honglei Yin, Offline Reinforcement Learning with Soft Behavior Regularization, NeurIPS 2022 offline RL workshop, Under Review, 2021. [Code]

Conference Proceedings

• Jianxiong Li$^*$, Jinliang Zheng$^*$, Yinan Zheng$^*$, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu, Jihao Liu, Yu Liu, Jingjing Liu, Ya-Qin Zhang, Xianyuan Zhan, DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning, International Conference on Machine Learning (ICML), 2024. [Code][Project Page]

• Xiao Hu$^*$, Jianxiong Li$^*$, Xianyuan Zhan, Qing-Shan Jia, Ya-Qin Zhang, Query-Policy Misalignment in Preference-Based Reinforcement Learning International Conference on Learning Representations (ICLR), 2024 (Spotlight). [Code]

• Yinan Zheng$^*$, Jianxiong Li$^*$, Dongjie Yu, Yujie Yang, Shengbo Eben Li, Xianyuan Zhan, Jingjing Liu, Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model” International Conference on Learning Representations (ICLR), 2024. [Code][Project Page]

• Jianxiong Li, Xianyuan Zhan, Haoran Xu, Xiangyu Zhu, Jingjing Liu, Ya-Qin Zhang, When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning, International Conference on Learning Representations (ICLR), 2023. [Code]

• Jianxiong Li$^*$， Xiao Hu$^*$, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Qing-Shan Jia, Ya-Qin Zhang, Mind the Gap: Offline Policy Optimization for Imperfect Rewards, International Conference on Learning Representations (ICLR), 2023. [Code]

• Haoran Xu, Li Jiang, Jianxiong Li, Zhuoran Yang, Zhaoran Wang, Victor Wai Kin Chan, Xianyuan Zhan, Offline RL with No OOD Actions: Offline RL with Implicit Value Regularization, International Conference on Learning Representations (ICLR), 2023 (Oral). [Code]

• Haoran Xu$^*$, Li Jiang$^*$, Jianxiong Li, Xianyuan Zhan, A Policy-Guided Imitation Approach for Offline Reinforcement Learning, Neural Information Processing Systems (NeurIPS), 2022 (Oral). [Code]

• Shiyue Zhao, Jianxiong Li, Xiao Hu, Junzhi Zhang, Chengkun He, Vehicle Extreme Control based on Offline Reinforcement Leaning, China Automation Congress (CAC), 2022.

📖 Educations

• 2021.08 - Present, PhD candidate, Institude for AI Industry Research (AIR) / School of Vehicle and Mobility, Tsinghua University, Beijing, China.
• 2017.08 - 2021.06, Undergraduate, School of Mechanical Engineering, Xi’an Jiaotong University, Shaanxi, China.

💻 Internships

• 2022.06 - 2023.05, Baidu, Beijing, China. Studing on the applications of offline RL algorithms on Traffic Signal Control (TSC).
• 2023.07 - 2023.09, IDRIVERPLUS, Beijing, China. Studing on the applications of life long learning algorithms on 2D object detection for autonomous driving.

🧑‍🎨 Services

Reviewer for NeurIPS 2023, ICLR 2024, ICML 2024, IJCAI 2024, DMLR workshop@ICLR 2024