AI-powered cartoonization has many practical applications these days — from personalized anime-style avatars to video and even fine art. Many black-box cartoonization frameworks however provide users with limited control or adjustability when rendering real-world photography into cartoon scenes. Now, researchers from ByteDance, The University of Tokyo and Style2Paints Research have introduced a framework that can generate high-quality cartoonized images with much-improved controllability in order to meet artists’ requirements across a wider range of styles and use cases.
Those who have experience with the cartoonization process will understand the difficulty involved in accommodating diverse cartoon styles and use cases, where task-specific assumptions or prior knowledge can be required to develop usable algorithms. Some cartoon workflows for example pay more attention to global palette themes and treat the sharpness of lines as a secondary issue. The artistic style in other workflows meanwhile may see sparse and clean colour blocks play a dominant role, with global palette themes relatively less emphasized. Black box cartoonization models struggle to effectively deal with such diverse workflow requirements, and using a black-box model to directly fit the training data can negatively affect generality and stylization quality, resulting in poor-quality outputs.
The researchers consulted artists and observed cartoon painting behaviour to identify three separate cartoon image representations: a surface representation that contains smooth surfaces, a structure representation that refers to sparse colour-blocks and flattens global content in the celluloid style workflow, and a texture representation that reflects high-frequency texture, contours, and details in cartoon images. Image processing modules extract each representation, and a generative adversarial network (GAN) framework is used to learn the extracted representations and cartoonize the input images. Output styles can be controlled by adjusting the weight of each representation in the loss function.
In a user study with 10 respondents and 30 images, the proposed approach outperformed three existing cartoonization methods in both cartoon quality (similarity between the input images and cartoon images) and overall quality (identifying undesirable colour shifts, texture distortions, high-frequency noise or other artifacts in the images).
The paper Learning to Cartoonize Using White-box Cartoon Representations is on GitHub. Click here to visit the project page.
Analyst: Grace Duan | Editor: Michael Sarazen; Fangyu Cai
Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors
This report offers a look at how the Chinese government and business owners have leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle.
Click here to find more reports from us.
0 comments on “New White-Box Framework Will Cartoonize Your World”