Shaoteng Liu

I am a Third-year PhD student at CUHK, advised by Prof. Jiaya Jia.

I got my B.Eng. degree at XJTU. I was a research assistant at the Berkeley Artificial Intelligence Research (BAIR) Lab, working with Dequan Wang.

My research interests mainly focus on image/video generation and editing currently.

Email / Google Scholar / Github

Selected Research

	Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, Jiaya Jia Preprint*, 2024 arXiv / Project Page / Demo / Model / Data / Code Mining potential of open-source VLMs! Mini-Gemini is a novel framework ranges from 2B to 34B VLMs for hi-resolution image understanding. It has a impressive OCR capability, and can generate HQ images powered by its multi-modal reasoning ability.
	Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, Qiang Xu ICLR, 2024 arXiv / Project Page / Video / Data / Code Rethinking the inversion process. Boosting Diffusion-based Editing with 3 Lines of Code.
	Video-P2P: Video Editing with Cross-attention Control Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, Jiaya Jia CVPR, 2024 arXiv / Project Page/ Code Add 'Lego' attribute to the child, an edited video is generated. Powered by a novel video inversion process and cross-attention control. We also find that a Decoupled-Guidance strategy is essential for video editing.
	MR-NeuS: Learning Neural Implicit Surfaces with Multiple Radiance Fields Shaoteng Liu, Sida Peng, Tao Hu, Xiangyu Zhang, Jiaya Jia Preprint, 2022 When adding an extra radiance field, the man sculpture becomes clearer.
	Self-supervised Learning by View Synthesis Shaoteng Liu, Xiangyu Zhang, Tao Hu, Jiaya Jia Preprint, 2022 arXiv A simple transformer is already able to synthesize high-quality images.
	On-target Adaptation Dequan Wang, Shaoteng Liu, Sayna Ebrahimi, Evan Shelhamer, Trevor Darrell Preprint, 2022 arxiv Target accuracy is the goal, so we argue for optimizing as much as possible on the target data.
	Tent: Fully test-time adaptation by entropy minimization Dequan Wang, Evan Shelhamer Shaoteng Liu, Bruno Olshausen, Trevor Darrell ICLR (Spotlight), 2021 arxiv/ Code Tent equips a model to adapt itself to new and different data during testing.
	Hyperbolic Visual Embedding Learning for Zero-Shot Recognition Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong-Wah Ngo, Tat-Seng Chua, Yu-Gang Jiang CVPR, 2020 Paper / Code Learn visual representation in hyperbolic space for Zero-Shot Recognition.