AI Computer Vision & Graphics Machine Learning & Data Science Research

Unselfie Turns Arm-Extended Selfies into Neutral-Pose Portraits

"Unselfie," a novel photographic transformation model that can automatically translate selfies into neutral-pose portraits.

The now-ubiquitous “selfie” started as a somewhat awkward photographic technique wherein the subject/photographer simply extends their phone camera away from their body to snap an in-situ shot. To avoid the telltale splayed arm selfie appearance, people need either a selfie stick, tripod and self-timer, or the assistance of another person to take the picture. But now, a group of researchers from KU Leuven, Adobe Research, and UC Berkeley has proposed “unselfie,” a novel photographic transformation model that can automatically translate selfies into neutral-pose portraits.

The team faced three main challenges in designing unselfie:

  1. No before-and-after photos were available for training purposes, so they had to train the model without such data.
  2. One selfie pose can potentially correspond to multiple neutral poses, thus they needed to teach the model to find the best substitution.
  3. How to maintain the overall picture quality by filling in the blanks in the background after changing the pose.
Screenshot 2020-08-04 09.16.46.png
Proposed three-stage pipeline

The researchers proposed a three-stage pipeline to address these challenges. They first synthesized selfie images from neutral-pose portraits using a non-parametric nearest-pose search module to retrieve the nearest selfie pose given a neutral-pose portrait and synthesize a corresponding selfie. They then applied the coordinate-based inpainting method to synthesize a coarse human body. The last step was the adoption of a gated convolutional layer-based composition network to jointly refine body appearance and fix background imperfections.

Because unselfie is novel research with no comparable prior work available, researchers modified two state-of-the-art human synthesis methods, Disentangled Person Image Generation (DPIG) and Pose-Attentional Transfer Network (PATN), to enable comparisons. The results show unselfie achieves sharper edges and significantly better restoration of clothing details.

Qualitative comparison results

The researchers also addressed problems with non-frontal selfie poses, which could result in the generation of overly wide or narrow arms. These issues however occurred in less than 10 percent of the results, and when they do, users are able to manually choose a better alternative from the top-5 results in the nearest pose search module.

Screenshot 2020-08-04 09.41.59.png
Arm distortion fails

The paper Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild is on arXiv.

Analyst: Reina Qi Wan | Editor: Michael Sarazen; Fangyu Cai

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon KindleAlong with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

1 comment on “Unselfie Turns Arm-Extended Selfies into Neutral-Pose Portraits

  1. Pingback: [R] Unselfie Turns Arm-Extended Selfies into Neutral-Pose Portraits : artificial - Sixpost News

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: