27 June 2024
Formulating the dynamics of continuously deformable objects and other mechanical systems analytically from first principles is an exceedingly challenging task, often impractical in real-world scenarios. What makes this challenge even harder to solve is that, usually, the object has not been observed previously, and the only information that we can get from it is a stream of RGB camera data.
In this work, EMERGE partners from the Delft University of Technology explore the use of deep learning techniques to solve this nonlinear identification problem. The authors specifically focus on extracting dynamic models of simple deformable objects from the high-dimensional sensor input coming from an RGB camera. They investigate a two-stage approach to achieve this goal. First, they train a variational autoencoder to extract an extremely low-dimensional representation of the object configuration. Then, they learn a dynamic model that predicts the evolution of these latent space variables. The proposed architecture can accurately predict the object's state up to one second into the future.
Read the paper in the link below.
More Information
