Unselfie Turns Arm-Extended Selfies into Neutral-Pose Portraits

The now-ubiquitous “selfie” started as a somewhat awkward photographic technique wherein the subject/photographer simply extends their phone camera away from their body to snap an in-situ shot. To avoid the telltale splayed arm selfie appearance, people need either a selfie stick, tripod and self-timer, or the assistance of another person to take the picture. But now, a group of researchers from KU Leuven, Adobe Research, and UC Berkeley has proposed “unselfie,” a novel photographic transformation model that can automatically translate selfies into neutral-pose portraits.

The team faced three main challenges in designing unselfie:

1. No before-and-after photos were available for training purposes, so they had to train the model without such data.
2. One selfie pose can potentially correspond to multiple neutral poses, thus they needed to teach the model to find the best substitution.
3. How to maintain the overall picture quality by filling in the blanks in the background after changing the pose.

The researchers proposed a three-stage pipeline to address these challenges. They first synthesized selfie images from neutral-pose portraits using a non-parametric nearest-pose search module to retrieve the nearest selfie pose given a neutral-pose portrait and synthesize a corresponding selfie. They then applied the coordinate-based inpainting method to synthesize a coarse human body. The last step was the adoption of a gated convolutional layer-based composition network to jointly refine body appearance and fix background imperfections.

Because unselfie is novel research with no comparable prior work available, researchers modified two state-of-the-art human synthesis methods, Disentangled Person Image Generation (DPIG) and Pose-Attentional Transfer Network (PATN), to enable comparisons. The results show unselfie achieves sharper edges and significantly better restoration of clothing details.

The researchers also addressed problems with non-frontal selfie poses, which could result in the generation of overly wide or narrow arms. These issues however occurred in less than 10 percent of the results, and when they do, users are able to manually choose a better alternative from the top-5 results in the nearest pose search module.

The paper Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild is on arXiv.

---

Analyst: Reina Qi Wan | Editor: Michael Sarazen; Fangyu Cai

---

---

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle. Along with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

---

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.