Hao-Yu (Max) Hsu

I am a first-year Ph.D. student in Computer Science at the University of Illinois Urbana-Champaign (UIUC), co-advised by Prof. Shenlong Wang and Prof. Alexander Schwing.

Prior to this, I obtained my M.S. degree from UIUC and my B.S. degree from National Tsing Hua University (NTHU). I was grateful to be selected as a Siebel Scholar during my Master's.

My research interests lie in 3D vision, generative modeling, and large multimodal models (VLM, MLLM). I am broadly interested in building computational systems that can perceive, represent, and interact with our physical world.

Email / CV / Google Scholar / Github / Twitter / LinkedIn

News

[2025.10] Awarded Amazon AI PhD Fellowship 2025-2026. 🎉
[2025.08] Started Ph.D. program in Computer Science at UIUC! 🌽
[2025.06] PhysTwin got accepted to ICCV 2025.
[2024.09] Awarded Siebel Scholar, Class of 2025. (USD $35,000) 🎉

Publications

* indicates equal contribution. Representative papers are highlighted.

HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video
Hongchi Xia, Chih-Hao Lin, Hao-Yu Hsu, Quentin Leboutet, Katelyn Gao, Michael Paulitsch, Benjamin Ummenhofer, Shenlong Wang
NeurIPS, 2025
arXiv / website / code /

We reconstruct simulation-ready and interactable digital twin assets from a single video.

PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos
Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, Yunzhu Li
ICCV, 2025
arXiv / website / code /

We optimize a spring-mass physics model of deformable objects and integrate the model with 3D Gaussian Splatting for real-time re-simulation with rendering.

AutoVFX: Physically Realistic Video Editing from Natural Language Instructions
Hao-Yu Hsu, Chih-Hao Lin, Albert J. Zhai, Hongchi Xia, Shenlong Wang
3DV, 2025
arXiv / website / code /

A system that generates dynamic, physically realistic visual effects (VFX) on a single video solely from text-based editing instructions.

SPoVT: Semantic Prototype Variational Transformer for Dense Point Cloud Semantic Completion
Sheng-Yu Huang*, Hao-Yu Hsu*, Yu-Chiang Frank Wang
NeurIPS, 2022
paper / website / video / code /

A point cloud semantic completion framework to complete partial point clouds of 3D objects with variational Transformer.

NeurMiPs: Neural Mixture of Planar Experts for View Synthesis
Zhi-Hao Lin, Wei-Chiu Ma*, Hao-Yu Hsu*, Yu-Chiang Frank Wang, Shenlong Wang
CVPR, 2022
arXiv / website / video / code /

Use an efficient 3D planar representation to model the geometry and appearance of a scene for novel view synthesis.

Experience

Research Assistant Vision & Learning Lab, National Taiwan University Sep. 2021 - Feb. 2023

Working with Prof. Yu-Chiang Frank Wang and Prof. Shao-Hua Sun on 3D vision and robot learning.

Awards & Professional Activity

Amazon AI PhD Fellowship, 2025–2026 [Amazon News]
Siebel Scholar, Class of 2025 [UIUC News] [Siebel Foundation News]
Phi Tau Phi Scholastic Honor Society Honorary Membership (Graduated with the first place in NTHU, Class of 2021)
Reviewer: CVPR, ECCV, SIGGRAPH Asia, 3DV, AAAI, ICRA, TPAMI.

Misc

I love various forms of sport activity. I often play basketball 🏀 in my spare time, and also enjoy baseball ⚾, swimming 🏊, weightlifting 🏋️, and cycling 🚴.

Template borrowed from Kaifeng Zhang and Jon Barron
Last updated: April 2026.