Virtually Try on Clothes with AI: A New Era in Online Shopping

Virtual try on has been around for years, but generative AI is making it considerably more realistic. Instead of relying on 3D garments placed on top of a person, new models can generate an image of what a garment might actually look like when worn.

One example comes from Google researchers with a model called TryOnDiffusion, introduced in the paper TryOnDiffusion: A Tale of Two UNets. The research has since contributed to Google's virtual try on experience, which lets shoppers see garments presented on models with different body shapes and skin tones. It addresses a simple but important problem with shopping for clothes online. A product photographed on one model tells you relatively little about how it might look on someone with a different body. In this post, we'll look at how the technology works and what it could mean for online fashion.

AI advances the level of realism

Traditional virtual try on experiences have often relied on augmented reality or 3D assets. These approaches can work well for products such as glasses, accessories and shoes, but clothing is considerably more difficult. Fabric folds, stretches, hangs and changes shape depending on both the garment and the person wearing it. Generative AI approaches such as TryOnDiffusion tackle the problem differently. Rather than placing a predefined 3D garment onto a body, the model generates a new image showing how the clothing could appear on that person. This can produce surprisingly realistic results without requiring every garment to first be recreated as a deta

Adapting to different bodies and poses

One of the biggest challenges in virtual try on is making the same garment work across different bodies and poses. Clothing needs to change shape naturally while still preserving details such as its cut, texture, pattern and proportions. TryOnDiffusion approaches this by generating the person and garment together. The model adapts the clothing to the pose and body of the model while attempting to preserve the characteristics of the original garment. This is particularly interesting for ecommerce. Instead of photographing every garment on multiple models, the same product image could potentially be used to visualize an item across a much wider range of bodies.

A different production model

Generative AI also changes how virtual try on content can be produced.Traditional 3D and AR solutions often require products to be recreated as 3D assets before they can be used in an experience. For a retailer with thousands of products and constantly changing collections, creating those assets can become a significant production task. AI based virtual try on can work directly from product photography, removing some of that preparation. The model generates the visualization when it is needed rather than relying on a manually created 3D version of every garment. This does not necessarily make AI faster than AR in real time. Generating an image with a diffusion model can require considerably more computation than displaying an existing 3D asset. The potential advantage is instead the amount of work required to get thousands of products into the experience in the first place.

Potential for full body try on

Early virtual try on solutions have often focused on individual product categories or upper body garments. Generative AI has the potential to expand this to complete looks, including trousers, dresses, outerwear and accessories.This becomes considerably more difficult as more of the body and outfit are involved. The system needs to understand how different garments relate to the body, to movement and to each other, while preserving the details of the actual products. If these problems can be solved reliably, virtual try on could eventually move from visualizing a single garment to showing complete outfits on different bodies.

AR + AI?

AI and AR do not necessarily need to compete. Combining the two could lead to more useful shopping experiences. AR is particularly good at creating interactive experiences that respond to the user and their environment in real time. Generative AI can create more realistic visualizations without requiring every garment to be manually recreated in 3D. Bringing these capabilities together could make virtual try on both more immediate and more visually convincing. There is still a major problem to solve, though. Looking realistic is not the same as predicting fit.

A generated image can show a convincing version of you wearing a jacket, but that does not necessarily tell you whether the sleeves will be too long, where the fabric will feel tight or whether you should order a different size. Solving that requires more than image generation. It requires reliable information about the person’s body, the garment’s dimensions, materials and how the fabric behaves. This is where current virtual try on experiences still have some way to go. Many existing applications are better at visualization, filters and brand engagement than answering the practical questions people have when buying clothes online.

But the direction is interesting. If virtual try on can eventually combine realistic visualization with reliable information about size and fit, it could become much more than an entertaining feature. It could actually help people make better decisions about what to buy.