Models¶
Models are grouped into families. Models in a family share a way of learning and a base class, so they can be swapped and compared on the same data, environments and planners. JEPA is the first family; more families and models will be added.
-
Joint-embedding predictive architectures: predict the representation of a target from the representation of a context, never its pixels. Conditioned on actions, they become latent world models you can plan with.
I-JEPA · LeJEPA · V-JEPA · VL-JEPA · LeWM · DINO-WM · RoboJEPA
Every model at a glance¶
| family | model | input | base class | notebook |
|---|---|---|---|---|
| JEPA | I-JEPA | images | JEPA |
vision/ijepa |
| JEPA | LeJEPA | images | JEPA |
vision/lejepa |
| JEPA | V-JEPA | video | JEPA |
vision/vjepa |
| JEPA | VL-JEPA | images or video + text | JEPA |
vqa/vljepa |
| JEPA | LeWM | frames + actions | WorldModel |
robotics/lewm |
| JEPA | DINO-WM | frames + actions + proprioception | WorldModel |
robotics/dinowm |
| JEPA | RoboJEPA | multi-view frames + actions + states | WorldModel |
robotics/robojepa |
Every model's forward returns a dict with a scalar "loss". Every WorldModel also has encode, predict and rollout, and plans with CEM out of the box. To add a model, see Build your own model.