Virtual Clothing Try-on Using Generative Adversarial Networks

Computer Science and Engineering

Virtual clothing try-on aims at transferring a target clothing image onto a reference person, and has become a major topic in recent years. The traditional try-on task aims to align the target clothing item naturally to the given person’s body and hence present a try-on look of the person. However, in practice, people may also be interested in their try-on looks with different poses. Therefore, in this work, we introduce a new try-on setting, which enables the changes of both the clothing item and the person’s pose. Towards this end, we propose a pose guided virtual try-on scheme based on the generative adversarial networks (GANs). We first predict semantic layout of the reference image that will be changed after try-on, and then determines whether its image content needs to be generated or preserved according to the predicted semantic layout, leading to photo realistic try-on and rich clothing details. In particular this involves three modules. First, a semantic layout generation module utilizes semantic segmentation of the reference image to progressively predict the desired semantic layout after try-on. Second, a clothes warping module warps clothing images according to the generated semantic layout, where a second-order difference constraint is introduced to stabilize the warping process during training. Third, a content fusion module that integrates all information (e.g. reference image, semantic layout, warped clothes) to adaptively produce each semantic part of human body. Experiments on our newly collected dataset demonstrate its promise in the image-based virtual try-on task over state of the art generative models.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.