Virtual Try-On Using Diffusion Models

A new paper from Alibaba’s research team explores what virtual try on could look like when it moves beyond still images.

ViViD, short for Video Virtual Try on using Diffusion Models, generates videos of a person wearing a different garment while preserving their movement and appearance. Rather than relying on 3D garments or AR overlays, the system uses generative AI to create the new footage.

At its core, ViViD uses diffusion models trained for video generation, combined with specialized encoders that understand both the garment and the person’s pose. Give the system a video of a person and an image of a garment, and it can generate new footage of that same person wearing it.

The difficult part is not simply putting a piece of clothing onto a body. It is making it behave consistently as the person moves.

Garment and pose understanding

ViViD separately analyzes the appearance of the garment and the movement of the person. This helps preserve details such as shape and texture while adapting the clothing to different poses throughout the video.

Consistency across frames

Video introduces a problem that static virtual try on does not have. The garment needs to remain coherent from one frame to the next. ViViD is designed to maintain both spatial and temporal consistency, keeping the clothing aligned with the body as the person moves.

Interesting potential for ecommerce

We’ve previously covered AI powered virtual try on and how generative AI could change the way fashion products are visualized online. Video makes that prospect considerably more interesting.

Retailers such as ASOS already use short videos to show how garments fit and move. If systems like ViViD become reliable enough, a retailer could potentially shoot a model once and generate product videos for multiple garments from the same source footage. That could dramatically change the amount of photography and video production required to put a collection online.

It could also make it easier to show products across a wider range of bodies and skin tones without having to reproduce the entire shoot for every variation.

The more ambitious possibility is personal try on. Instead of looking at a model wearing the product, you could upload a short video of yourself and see how different items might look on your own body, in motion.

ViViD is still research rather than a finished ecommerce product, but it points towards a version of virtual try on that feels much closer to how we actually judge clothing. Not just how it looks in a single image, but how it looks when someone moves.

Read the paper here.