Charlot Lab · CORE · phase 2 spike (internal)

Planning in a learned latent world model

Phase 1 showed the sparse latent can hold a memory. Phase 2 is the hard question: can that same sparse latent be a JEPA world model, an encoder plus an action-conditioned predictor, trained to forecast the next latent, that you can plan in? Here CORE learns a grid world from random play (no rewards, no labels), predicting only in its own 32-dim latent. Then it reaches a goal by energy descent in that latent, at each step it imagines all four moves and takes the one whose predicted latent is closest to the goal's. It never sees the real dynamics while planning; only its learned model.

obs→encoder → sparse latent x→predictor x̂=P(x,action)→energy = ‖x̂ − x_goal‖→act = argmin energy

Learning the world from random play (training encoder + predictor in your browser)…