AI Machine Learning & Data Science Research

BIGO and iQIYI’s ClothFormer: Realistic Video Virtual Try-on Come True

A research team from BIGO Technology and iQIYI Inc. presents ClothFormer, a novel video virtual try-on framework that preserves clothes’ and humans’ features and details to generate realistic and temporally smooth try-on videos that surpass the outputs of current state-of-the-art virtual try-on systems by a large margin.

Total global retail e-commerce sales have more than tripled over the last six years and are projected to top US$7 trillion by 2025. With fashion claiming an increasing share of this market, suppliers are increasingly deploying AI-powered virtual try-on systems. Such systems are not only changing buyers’ shopping habits and boosting the e-commerce industry, they also have applications in short video and other popular domains. While the quality of image-based virtual try-on methods has dramatically improved, video-based virtual try-on remains relatively underdeveloped, as it is difficult and computationally costly to generate visually pleasing and temporally coherent video results.

In the new paper ClothFormer: Taming Video Virtual Try-on in All Module, a research team from BIGO Technology and iQIYI Inc. presents ClothFormer, a novel video virtual try-on framework that preserves clothes’ and humans’ features and details to generate realistic and temporally smooth try-on videos that surpass the outputs of current state-of-the-art virtual try-on systems by a large margin.

The team summarizes their main contributions as:

  1. A novel warp module that combines the advantages of TPS-based methods and appearance-flow-based methods is designed to address the problem of inaccurate warp due to occlusions appearing in the clothing region.
  2. A tracking module based on ridge regression and optical flow correction is proposed to deform a temporally smooth warped clothing sequence, which provides a prerequisite for the try-on module to generate coherent videos.
  3. The MPDT generator is designed carefully in the try-on module, which can extract and fuse clothing textures, person features and environment information to generate realistic try-on videos. To the best of our knowledge, this is the first time that transformer has been applied to video virtual try-on.

The main limitations of existing video virtual try-on methods are their poor performance with regard to frame consistency and spatio-temporal smoothness. The researchers trace these problems to two factors: 1) Existing models focus too much on the try-on module while neglecting the spatio-temporal dimensions, which leads to blurring and temporal artifacts in the generated videos; and 2) Most models were trained on simple datasets with clean backgrounds, and thus struggle in more complex real-life environments.

The proposed ClothFormer aims at solving the abovementioned issues. The team first designs a clothing-agnostic person representation that eliminates any clothing information and preserves backgrounds and occlusions. They then employ a frame-level TPS-based warp method to predict and mask the clothes’ occlusion regions, and feed these predicted results to an appearance flow-based method to obtain accurate and anti-occlusion dense flow pairs between the body and clothing regions. They also use an appearance flow tracking module to obtain warped clothing sequences with improved spatio-temporal consistency. Finally, they introduce a novel Multi-scale Patch-based Dual-stream Transformer (MPDT) generator, which extracts and fuses clothing textures, person features such as pose, and environment information to synthesize the final output video sequence.

To validate the effectiveness of the proposed ClothFormer, the team compared its outputs with existing state-of-the-art methods (FW-GAN, MV-TON, CP-VTON, ACGPN and PB-AFN) on both image-based and video-based evaluation metrics on the VVT video virtual try-on dataset. In the experiments, ClothFormer achieved significant quantitative and qualitative improvements in high-quality and spatio-temporally consistent try-on video generation, surpassing current systems by a large margin.

Demos in video format are available on the project’s GitHub. The paper ClothFormer: Taming Video Virtual Try-on in All Module is on arXiv.


Author: Hecate He | Editor: Michael Sarazen


We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.

23 comments on “BIGO and iQIYI’s ClothFormer: Realistic Video Virtual Try-on Come True

  1. Canon printers are mostly preferred by professionals who are into the business of photo printing. Canon printers produce high-quality prints and have an easy ink refill process. Canon printers perform faster than other printers. In order to perform the setup procedure for your Canon printer, you can reach the website ij.start.canon setup right now.

  2. Canon printers are highly reliable and perform excellent printing performance. Their print quality is awesome. Whether you are in search of an inkjet printer, or laser printer, Canon manufactures them both. Canon printers are an excellent choice for professionals wishing to get high-quality printouts and speedy performance. Set up your Canon printer through ij.start.canon set up now.

  3. Cricut cuts leather, card stock, sticky paper, and other materials into shapes. It can make crafts for you by cutting hard paper into greeting cards and labels. A Cricut machine buyer can also give an attractive shape to vinyl by cutting it with precision. Reach cricut.com/setup to set up your Cricut machine now.

  4. Canon is a prominent printer brand and has received numerous accolades from people worldwide. Canon printers make inkjet and laser printers. Whether you want to print, copy, or scan documents, you will find canon printers quite perfect for your printing needs. In order to perform the setup procedure for your newly bought Canon printer, you can get to the official site ij.start.canon set up right now.

  5. Silhouette Cameo is your chance to create a world of arts and crafts with the maximum possible designs. In addition, you can get unlimited sets of patterns as well. If you need to create eye-catching art pieces every day, the Silhouette machine is the only one. It is the only electronic cutting machine that can help you cut various types of materials accurately without putting any effort. The shape and size feature a resemblance to your home printer.

  6. Silhouette Cameo is a great desktop cutting machine that lets you create crafts in an easy way. You can set it up at your home and quickly start the process of making crafts. It cuts materials with precision according to the commands sent by you. It lets a user cut fabric stickers and create custom stickers and tattoos. Moreover, It can cut materials such as leather, vinyl, cardstock, paper, etc. Carry out the Silhouette Cameo setup process to begin using your Silhouette Cameo machine right away.

  7. Cricut is a machine that beautifully prints and cuts crafts for you. If you need a perfect machine that works as per your instructions and provides designs according to your requirement for several school projects or commercial work, then Cricut is the best solution for you. Cricut.com/setup comes with multiple tools that help you to make a large design in a few minutes. Still, if you have not bought the Cricut machine yet and need one, then purchase it right now from the online store or an offline market.

  8. Setting up your Rollo printer has now become fast and effortless. The Rollo printer’s product page is waiting for you; just click rollo.com/setup. Choose your printer, Wired or Wireless. Then check the in-depth installation video. After doing that, install the printer driver. And there you go, your printer is ready. It is as simple as we made it sound, even more than that. Click the highlighted link and get started.

  9. For multifunctional printers, you can easily get the setup directions at ij.start.canon If your query is about how to connect the Canon printer to Wi-Fi, this guide is for you. Get the instructions whether you require help with wireless setup or connect your printer model to Wi-Fi. Also, you explore ij.start.canon to get the driver for your particular Canon printer series.

  10. Your Cricut machine allows you to create something out of the ordinary. Click Cricut.com/setup and get a chance to be your own DIY craft master. Before using your model, you will need to set it up properly. Also, you will have to understand what exactly you can use Cricut for.

  11. Charlie Flint

    I believe that it can actually boost sales and help entrepreneurs in that field. However, it’s important to remember about other aspects of running that business, and understanding the Buyer persona is one of them. It can be complicated at first, but it can be highly effective in terms of sales as well.

  12. Gaven Rank

    so dope really

  13. If you are faced with the fact that you do not know where to buy just great women’s clothes. I myself once looked, but now there is no problem with it. Look just here Online shopping is continuing to grow in the EU https://inlower.com/womens and choose what you need for your image, style, size, and so on. There are a lot of different so that everyone for themselves what they will find. So I recommend that you check it out. I hope that helped.

  14. Cricut Joy is the perfect crafting companion for people on-the-go. This compact and portable cutting machine can fit in your bag and is ideal for small projects, such as scrapbooking. Cricut Joy is also compatible with a range of smart materials, including paper, vinyl, and cardstock, and does not require a cutting mat. With Bluetooth connectivity, you can easily control the functions of Cricut Joy from anywhere. The Design Space app allows you to create or edit designs and send them directly to Cricut Joy via Bluetooth. To download the app, simply visit design.cricut.com/setup to download the app.

  15. You can prepare satisfying DIY projects with the help of a Cricut machine. The Cricut produces different machines for your basic needs, such as writing, engraving, heating, debossing, scoring, etc. To use a Cricut machine, you will need to connect your machine to a compatible device like a smartphone, laptop, or iOS. Also, Design Space Software is crucial to operating your Cricut machines (Cricut Maker & Cricut Explore). So, visit cricut.com/setup to download and install Design Space.

  16. To set up your Explore and Maker . Get started with Cricut Design Space , you can follow these steps: for Setup Cricut explore or Maker visit: Cricut.com/setup

  17. The cricut maker is an advanced smart cutting machine that offers an extensive range of tools, materials, and possibilities compared to other craft plotters. With the ability to cut over 300 different materials, including Smart Materials, it is perfect for creating various 3D projects like school projects and home Decor. To connect your devices, you can visit mycricutmaker.com. This comprehensive guide provides all the information you need regarding the connection to your cutting machine and the download process for the Cricut software/app.

  18. With its sharp blades, tools & other accessories, the crafter gets the golden opportunity to cut down different kinds of materials into the shape of their choice. This smart cut-die machine holds the ability to thinnest to thickest materials within a few minutes. To enjoy the benefits of this machine, you need to register from design.cricut.com

  19. A Cricut machine is incredibly user-friendly and easy to use; its inherent interface is easy-to-use software. You can simply connect the machine to your computer or mobile device and start designing and cutting in no time. The software has a vast library of designs, patterns, images, and fonts so that you can create your DIY project with one click. To download the software, you can visit cricut explore setup.

  20. The DIYer shouldn’t leave the golden chance of transforming their DIY ideas into reality. In this scene, Design Space plays a perfect role for every crafter spread across the world. This easy-to-use software offers unlimited images, fonts, and ready-to-make projects, which helps the DIYer in creating different projects. To enjoy the benefits of this software, the crafter can install it from cricut.com/setup.

  21. If you are a Cricut crafter or planning to buy a Cricut cutting machine, then it is to create an account. To create an account, crafters can simply browse the install cricut design space app website and select the Create Cricut ID option. A Cricut account helps with browsing the website, setting up the machine, and others. Plus, an account helps sign in to your Cricut Design Space app, which helps sync the app on multiple devices. In addition, without a Cricut account, crafters will not be able to set up and use the Cricut machine or Design Space app. Browse the website and start the DIY crafting passion now.

  22. Cricut Marts is a team of Cricut experts who specialize in solving issues related to all Cricut machines, including Cricut Maker, Maker 3, Joy, Explore Air 2, Heat Press, Mug Press, and more. Cricut™ makes smart cutting machines that work with an easy-to-use app to help you design and personalize almost anything.

  23. How to login Bitdefender central Account Page . login Bitdefender central with the same keyword and and visit our official website for login you we need some of general information which you fix when you created Bitdefender login enter your mail and your required things which have been saved with us.

Leave a Reply to ij.start.canon setup Cancel reply

Your email address will not be published. Required fields are marked *