Video summary

Нурислам Зиннатуллин, Амир Нигматуллин, Александра Вабниц | От паттерна до продакшена

Main summary

Key takeaways

Technology

Summary of the video (tech/product overview + how it works)

The speakers explain how their company uses generative AI to create realistic interior photography, specifically generating curtains and flooring images for a web catalog. The goal is to reduce time and cost compared with traditional workflows (photoshoots + 3D rendering + manual editing).

They position GenAI as more than text/code: it’s also about image generation and control to help designers and improve the website’s visual content.

Business/product goals

  • Speed up when new photos appear on the website.
  • Lower cost by reducing reliance on photographers and missing supplier assets.
  • Enrich the customer journey with better interior visuals.
  • Increase organic traffic (mentioned as a goal; metrics aren’t provided yet because it’s still being tested).

User workflow (from the demo)

Target users

  • Mainly designers, using a front-end UI.

Typical flow

  1. Designers input an article/product code.
  2. The system:
    • Selects a pattern (for curtains or tile/board materials).
    • Shows original vs generated results for comparison.
    • Lets users tune parameters (e.g., fabric textures, shadows, shine, color correction).
    • Enables saving drafts, testing multiple options, and choosing the best.

Flooring UI details

  • Choose:
    • room
    • style
    • number of pictures
  • Mask configuration:
    • A highlighted mask (green) can be refined if its quality is poor.
  • Users can download the final generated results.

Product scope currently live (after initial testing)

Two AI-enabled categories are integrated and ready for designer testing:

  • Curtains
  • Floor coverings

Other categories were tested earlier (e.g., lighting/small decor), but designers prioritized curtains and floors. Expansion is planned, but details aren’t provided.

Technical approach: two main pipelines

Pipeline 1 — “Rendering on a template” (Curtains; simpler)

Goal

Generate a curtain in a fixed template interior using classical CV/rendering steps plus learned parameter prediction.

Inputs

  • Pattern/material image
  • Fixed binary mask defining object boundaries (curtain region)
  • User/control parameters (position, texture, shadows, gloss)

Generation method (“under the hood”)

This pipeline uses classical computer vision / render-style steps rather than a neural network for the base transfer:

  • Pattern scaling/stretching to match placement
  • Compute gradients along the curtain to infer fold locations
  • Compress texture along gradients to simulate folds
  • Cut using the mask and add shadows from the template
  • Produce a mixture/light-channel composition

Optional post-processing adjusts shine/shadow characteristics.

Automating the control parameters

They trained a dataset mapping:

  • images + template patterns + masks → good parameter sets

A relatively simple ML setup predicts parameters:

  • Architecture uses DIN-based embeddings (two embeddings per image, concatenated)
  • A regression head predicts ~15 numeric parameters (e.g., gloss intensity, brightness correction)
  • A classification head predicts ~4 binary parameters (e.g., whether to add a velvet effect)

Pipeline 2 — “Generator + mask + transfer” (Curtains + Floors; more complex)

Goal

Generate an entire interior scene, then place the product material into it using masks.

Differences vs Pipeline 1

  • Replace fixed template with a generated interior
  • Replace fixed mask with a mask derived from the generated interior

Scene generator

  • Uses FLUX.1 (via FLX Crew mentioned) tuned for realistic interior photographs.
  • Key requirements:
    • realistic output
    • diversity
    • controllable variations (room types, styles, color schemes)

To address limited product variety, they build long prompts from many small parts (“prompt dictionaries”), tuned per category by rewriting prompts.

They report strong prompt adherence “out of the box,” but needed engineering to get enough diversity for one product.

Segmentation subsystem (masking objects)

Two-stage masking:

  1. YOLOv11 detector (instance detection; produces object boxes for curtains/tulle/floor regions)
  2. SAM v2 segmenter (refines masks inside those boxes)

Both YOLO and SAM variants were trained using manually labeled images. They report good mask quality using Intersection-over-Union (IoU).

Material placement

For each category, they generate:

  • the interior backdrop
  • a segmentation mask
  • then apply a pattern transfer algorithm to integrate the curtain/floor material realistically

Flooring-specific pipeline details (extra stages)

The floor pipeline adds realism steps beyond the two-stage generator/mask/transfer idea:

  1. Pattern preparation

    • Supports board types: single and multiple
    • For multiple boards: segment boards, cut segments, and color-balance to fit together

  2. Interior generation (with constraints)

    • Emphasizes the flooring area
    • Floor is required to be strictly white, with controlled finish characteristics (no matte/gloss note is mentioned as part of the constraint)

  3. Floor mask selection

    • Additional training of SAM for the floor category

  4. Perspective “testing ground”

    • Create perspective by placing a black grid on the white floor
    • The grid helps the model infer tilt so boards look realistic

  5. Assembly

    • Supports two installation patterns:
      • half offset
      • diagonal offset
    • The user selects the style during generation

  6. Final compositing

    • Combine generated interior + assembled floor
    • Apply color correction, light correction, and align white/black balance for photo-quality output

“Productization” & production/infra notes (timing + deployment)

  • The backend is split into components to manage GPU-heavy workloads:
    • ML endpoints run separately for easier resource control
  • UI/backend data sources include:
    • S3, databases, and media storage

Model-serving/deployment flow

  • Uses containerized builds and CI/CD with Jenkins:
    • Jenkins watches directories → builds Docker images → pushes to Artifactory → deploys using config
  • Model weights and generated outputs use secrets via Vault and S3

They also describe endpoint behavior (replicas, resource requirements, response times).

Total end-to-end ML pipeline time

  • Curtains: ~30 seconds
  • Floors: ~50 seconds

UX detail

  • It’s not a single synchronous wait. Users configure results via step-by-step stages, making the process feel smoother than an all-at-once pipeline.

Evaluation & testing commentary (including “Turing test” style feedback)

They performed informal tests where people tried to guess whether images were generated.

  • Curtains: viewers sometimes couldn’t reliably distinguish generated vs real (“Turing test passed for now”).

  • Floors: some people were more likely to over/under-estimate authenticity depending on the viewpoint, but overall the speakers claim the generated interiors look good.

They note occasional generation flaws, but say desired results are reached “to some extent.”

Questions from the audience → key answers

  • Availability beyond designers: currently designer-focused; no broad rollout yet.
  • Metrics for the traffic goal: no quantified metrics yet (still in testing).
  • End-to-end diffusion/VLM with material insertion: tested approaches (e.g., Stable Diffusion/FLUX with adapters/controlnets; open models) struggled with texture preservation, which is critical for curtains/floors realism—classic CV transfer performed better.
  • Diversity vs sameness: solved with large prompt dictionaries (many prompt parts like angle/composition/atmosphere with many variations). They report enough diversity across ~100 generations.
  • Scaling to other home accessories: other categories exist but were deprioritized; focus remains on curtains/floors for now.
  • Collaboration for pre-build interior customization: no clear plans; more likely for individual user customization. They’re also building their own 3D-room product where users configure objects.
  • Automated constructors vs designer UI: the current solution focuses on catalog/supplier content completion (filling missing interiors), not a full builder/wallpaper-shop style constructor experience. Future automation for full interior selection is implied but not committed.
  • Notable testing bug (humorous): an example where a jar with herring/jam was generated with incorrect product labeling (jam vs herring).

Main speakers / sources

  • Nurislam Zinnatullin / Zinatulin (also speaking first; overview and productization/ML split)
  • Amir Nigmatullin / Nigmatulin (flooring pipeline features)
  • Alexandra Vabnets / Vabunits (curtains category technical approach; templates/pipelines, segmentation, prompt engineering)

Audience members contributed Q&A, but the technical content primarily comes from the three speakers above.

Original video