Inverse-Foley Animation: Synchronizing rigid-body motions to sound

In this paper, we introduce Inverse-Foley Animation, a technique for optimizing rigid-body animations so that contact events are synchronized with input sound events. A precomputed database of randomly sampled rigid-body contact events is used to build a contact-event graph, which can be searched to determine a plausible sequence of contact events synchronized with the input sound's events. To more easily find motions with matching contact times, we allow transitions between simulated contact events using a motion blending formulation based on modified contact impulses. We fine tune synchronization by slightly retiming ballistic motions. Given a sound, our system can synthesize synchronized motions using graphs built with hundreds of thousands of precomputed motions, and millions of contact events. Our system is easy to use, and has been used to plan motions for hundreds of sounds, and dozens of rigid-body models.

• Paper (16M)
• Supplemental Video [high quality (196M)]
• SIGGRAPH 2014 Keynote Slides [zip (235M)]

Timothy R. Langlois and Doug L. James. Inverse-Foley Animation: Synchronizing rigid-body motions to sound, ACM Transactions on Graphics (SIGGRAPH 2014). 33(4), August, 2014.

• Extreme Tech - Siggraph 2014: Where the world's top computer graphics experts share their latest research on realistic melting bunnies

• The National Science Foundation (HCC-0905506, DGE-1144153)
• The John Simon Guggenheim Memorial Foundation
• Intel (ISTC-VC)
• Pixar Animation Studio
• Autodesk
• Mitsuba Renderer