03Embodied navigation具身导航

How can an embodied agent follow long-horizon instructions in unseen environments and on real robots?

Vision-and-language navigation requires an agent to track its progress through an instruction, ground the landmarks that matter for the current subgoal, and understand the geometry around them before it acts. At AgiBot X-Lab I lead this research for the Lingxi X2 and A3 Ultra humanoids.

GuidedNav shapes a policy's pre-action attention with subtask, landmark and spatial guidance that is used only during training, and reaches state-of-the-art success among compared monocular RGB policies on R2R-CE and RxR-CE. NavTTT gives the policy a parametric memory updated by hierarchical test-time training, so it keeps long-horizon history beyond its context window. HUGE-Bench extends evaluation to high-level UAV tasks in digital-twin scenes, and ExecVLA makes vision-language-action models follow fine-grained execution constraints.

Papers

S16

GuidedNav: Shaping Vision-Language Navigation Representations through Pre-Action Attention and Spatial Guidance

Jingyu Guo, Jiaxin Huang^†, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong

arXiv 2026, under review · project leader

arXiv Project page Code

A unified framework that shapes the representations connecting instructions to actions through training-time guidance. Subtask and landmark guidance steer pre-action attention toward the active instruction segment and the relevant visual regions; spatial guidance aligns internal representations with geometric features. Among compared monocular RGB policies, GuidedNav raises success rates by 8.1% on R2R-CE and 13.1% on RxR-CE val-unseen with faster inference, and releases the Guided-R2R and Guided-RxR datasets.

GuidedNav overview: subtask guidance (what to do), landmark guidance (what to attend to) and spatial guidance (how to move in 3D) along a robot's route through a house, with success-rate and speed comparisons. — TeaserThree kinds of guidance along one route: which subtask comes next, which landmark to attend to, and how to move in 3D.

GuidedNav architecture: a large language model reads instruction, history and current visual tokens; training-only modules guide attention at the pre-action position toward the instruction subtask, the landmark in the current frame, and spatial features. — FrameworkGuidance acts at the pre-action position and only during training: subtask attention over the instruction, landmark attention over the current frame, and long-horizon and local spatial guidance. At inference the policy runs without these modules.

S15

NavTTT: Adaptive Parametric Memory via Hierarchical Test-Time Training for Vision-and-Language Navigation

Xiaoyu Luo, Chengxing Lin, Jiaxin Huang^†, Jiale Liu, Jingyu Guo, Sirui Chen, Aolin Li, Shaoli Huang, Cheng-zhong Xu, Hui Kong

Under review · project leader

An adaptive parametric memory built by hierarchical test-time training. The navigation policy keeps long-horizon history beyond its context window and follows instructions on a real robot.

HUGE-Bench: A Benchmark for High-Level UAV Vision-Language-Action Tasks

Jingyu Guo, Ziye Chen, Ziwen Li, Zhengqing Gao, Jiaxin Huang, Hanlue Zhang, Fengming Huang, Yu Yao, Tongliang Liu, Mingming Gong

ECCV 2026

arXiv Project page

A benchmark for high-level UAV vision-language-action tasks: 4 real-world digital-twin scenes, 8 high-level tasks and 2.56M metres of trajectories on an aligned 3DGS-mesh representation, with process-oriented and collision-aware metrics.

HUGE-Bench overview: four drone tasks (building inspection, mapping mission, roadway inspection, traversal mission) drawn as flight paths over real-world digital-twin scenes, around a panel listing 4 scenes, 8 high-level vision-language-action tasks, 2.56 million metres of trajectories and a collision-aware simulator. — TeaserHigh-level drone tasks such as building inspection, mapping and traversal, set in four real-world digital-twin scenes with a collision-aware simulator.

ExecVLA: Following Fine-Grained Execution Constraints in Vision-Language-Action Models with Bi-Level Action Representation

Gaoge Han, Zhengqing Gao, Ziwen Li, Jiaxin Huang, Shaoli Huang, Fakhri Karray, Mingming Gong, Tongliang Liu

arXiv 2026

arXiv

A vision-language-action policy split into goal-level and execution-level components through a bi-level action representation and supervised reasoning tokens, so that it follows fine-grained execution constraints such as the interaction target, the motion pattern and spatial relations, not only the goal.

山重水复疑无路Lu You · Aristotle

East · 东

Hills upon hills, streams upon streams: I doubt there is a way through. Then shady willows, bright flowers, and another village.

Lu You 陆游, “Visiting a Village West of the Mountain” 《游山西村》, 1167

山，重，水，复，疑，无，路，柳，暗，花，明，又，一，村。

West · 西

For the things we have to learn before we can do them, we learn by doing them.

Aristotle, Nicomachean Ethics II.1, 1103a (tr. W. D. Ross)