HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks

ICLR 2021

Zhou Xian, Shamit Lal, Hsiao-Yu Fish Tung, Emmanouil Antonios Platanios, Katerina Fragkiadaki
Carnegie Mellon University


HyperDynamics encodes the visual observations and a set of agent-environment interactions and generates the parameters of a dynamics model dedicated to the current environment and timestep using a hypernetwork. HyperDynamics for pushing follows the general formulation, with a visual encoder for detecting the object in 3D and encoding its shape, and an interaction encoder for encoding a small history of interactions of the agent with the environment.


Abstract

We propose HyperDynamics, a dynamics meta-learning framework that conditions on an agent’s interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal dynamics are inferred from the interaction history and visual observations, and are implicitly captured in the generated parameters. We test HyperDynamics on a set of object pushing and locomotion tasks. It outperforms existing dynamics models in the literature that adapt to environment variations by learning dynamics over high dimensional visual observations, capturing the interactions of the agent in recurrent state representations, or using gradient-based meta-optimization. We also show our method matches the performance of an ensemble of separately trained experts, while also being able to generalize well to unseen environment variations at test time. We attribute its good performance to the multiplicative interactions between the inferred system properties---captured in the generated parameters---and the low-dimensional state representation of the dynamical system.



Experimental Results
We evaluate HyperDynamics and the baselines on prediction error for both single-step (t = 1) and multi-step unrolling (t = 5) (Table 1). We also test their performance for downstream manipulation tasks, where the robot needs to push the objects to desired target locations with MPC, in scenes both with and without obstacles. When obstacles are present, the agent needs to plan a collision-free path. We report their success rates in 50 trials in Table 2. Red numbers in bold denote the best performance across all models, while black numbers in bold represent the oracle performance (provided by an ensemble of separately trained experts). Our model outperforms all baselines by a margin on both prediction and control tasks.



For locomotion tasks, we consider a legged robot walking in a dynamic environment with changing terrains, where their properties are only locally consistent. We apply all the methods with MPC for action selection and control, and report in Table 3 the average return computed over 500 episodes. Again, red numbers denote the best performance and the black ones represent the oracle performance. In all tasks, HyperDynamics outperforms all the baselines. It is able to infer accurate system properties and generate corresponding dynamics models that match the oracle Expert Ensemble model on seen terrains, and shows a great advantage over it when tested on unseen terrains.


ICLR 2021 Talk



Paper


Code