❯ Yann LeCun’s team publishes H-JEPA, a hierarchical world model whose three-level version lifts maze success from 18% to 73%

Planning in layersYann LeCun and four co-authors posted the paper H-JEPA on arXiv on October 5. It proposes a hierarchical world model that lets robots plan over long horizons from visual input. A world model predicts what the environment will look like after a given action. According to 36Kr, the work comes from researchers at AMI, the company LeCun founded, with New York University, INRIA Paris and Brown University.

Top levels set direction, bottom levels actH-JEPA stacks several JEPA models. No level generates images; each predicts the future in its own learned abstract representation, and higher levels predict further ahead. Planning runs top-down: the top level sets the rough route toward the goal, its prediction becomes a subgoal for the level below, and this is refined level by level until the bottom one outputs actions. The paper says higher levels drop fast, unpredictable detail and keep slower, task-relevant state.

Results in the mazeAcross four simulated navigation and manipulation environments, hierarchical planning beat a single-level JEPA. On the Visual AntMaze task, a three-level hierarchy raised success from 18% to 73% while using less planner compute. More levels are not always better: 36Kr reports that two levels worked best on the Push-T task and that results fell with three or four. The team has released its code and pre-trained weights.

Still some distance from real robotsThe paper also tested the approach on DROID, a dataset of real-robot videos, where hierarchy improved offline planning accuracy. That is an offline evaluation, not a robot completing tasks. AMI raised $1.03 billion in March and has no public product yet. This paper is one of its few public technical results and gives outsiders a way to judge how far a route that avoids large language models has come.

▮ SIGNALLong tasks are hard because errors pile up step by step, and this paper’s answer is to set direction on a coarser timescale first and break it down from there, not to have the model try one step at a time.