Computer scientists at the University of Washington (UW) have found a way to make every Harry Potter fan's dream come true by bringing two-dimensional photos to life. (Sorry kids, the magic is in the technology and not the wizarding world.)
An algorithm called Photo Wake-Up uses augmented reality tools to bring animated movement to an object from a two-dimensional photo or piece of artwork by giving them the ability to walk, run, and even jump.
“This is a very hard fundamental problem in computer vision,” said study co-author Ira Kemelmacher-Shlizerman in a statement. “The big challenge here is that the input is only from a single camera position, so part of the person is invisible. Our work combines technical advancement on an open problem in the field with artistic creative visualization.”
Moving images of superhero Ironman, basketball superstar Steph Curry, and even the abstract central characters in famous Picasso and Matisse artworks highlight the algorithm’s ability to bring still images to life. It works by first identifying the person or object in an image by outlining their body to create what researchers call a “mask”. A 3D template is then matched to and wrapped around the body position so that the object takes on a multi-dimensional role over a two-dimensional background. Information is stored in each pixel to track the distance between different points on the object’s body for fluid motion that is maintained when the algorithm then puts the original texture (colors and patterns) back over the under wrapping.
It was once thought impossible, but the researchers write in their study, available to read on pre-print site arXiv, that their work can help children bring their artwork to life, add depth to museum exhibits, and even provide realistic approaches to video game experiences.
“There is some previous work that tries to create a 3D character using multiple viewpoints,” said co-author Brian Curless. “But you still couldn’t bring someone to life and have them run out of a scene, and you couldn’t bring AR into it. It was really surprising that we could get some compelling results with using just one photo.”
Photo Wake-Up currently works best with forward-facing images. The researchers will be presenting their work at the annual Conference on Computer Vision and Pattern Recognition in southern California later this month.