Why LightX2V is fast¶
LightX2V's speed is not one kernel toggle. It comes from deleting work, specializing the execution graph for each model family, moving fewer bytes at the right time and then selecting hardware-aware operators. Vflash preserves that architecture and adds reproducible profiles, service boundaries and evidence.
Layered execution model¶
- Model-specific semantics. The H3 runner understands H3's packed video, audio, text and reference-token layout instead of forcing it through a generic diffusion abstraction. H3 uses a dense single-stream Omni-Transformer and separate video and audio VAEs; the official architecture is documented in the MiniMax H3 repository.
- Avoided duplicate work. Turbo profiles run without CFG, so they do not execute a second unconditional branch. Exact AdaLN values can be precomputed, matching the official observation that roughly 13B of H3's 33B parameters reside in AdaLN-related branches and do not all need to remain resident for inference-only deployment.
- Block-granular weight streaming. CPU offload uses small execution blocks rather than moving an entire model phase at once. Two reusable device buffers allow the next block to prefetch while the current block computes, reducing serial PCIe waiting.
- Operator registry. Attention, RMS normalization and RoPE are selected per profile. The measured consumer-GPU path uses FlashAttention3 or SageAttention2 plus strict Triton RoPE where the exact shape and numerical contract have passed checks.
- Precision and cache choices. BF16 is the reliable baseline. INT8, FP8 and caching only advance when their end-to-end speed survives video, motion and audio checks; a fast matrix multiply alone is insufficient.
- Parallelism matched to the goal. Tensor parallelism reduces one request's latency at a communication cost. Independent resident workers increase batch throughput without that cross-GPU traffic. The two numbers are reported separately.
- Resident service lifecycle. Model construction, tokenizer loading and memory allocation stay outside the request critical path. The API serializes access to one pipeline and fails closed after fatal CUDA/OOM errors so stale state is not reused.
Vflash additions to the pinned upstream¶
| Area | Upstream capability | Vflash contract |
|---|---|---|
| Revision | moving main |
exact LightX2V commit in configs/upstreams.toml |
| RoPE | Triton implementation | strict H3 temporal/spatial rounding patch |
| Reference resize | framework behavior | explicit match, max and diffusers; match default |
| NFE | Base and distilled paths | named Base24/40/50 and Turbo4 profiles |
| Cache | framework cache primitives | request-scoped Pattern3 warmup20 with fail-closed reset |
| Quantization | multiple backends | selective hybrid INT8 patch remains experimental |
| Service | CLI/pipeline building blocks | resident FastAPI worker with bounded request contract |
| Topology | framework parallel flags | public-safe topology snapshot and explicit single/TP2 launch |
The patches are deliberately small and checked against the pinned source. They do not fork model weights or hide upstream provenance.
Bottleneck-first optimization¶
The critical path changes after optimization. Base40 spent 93.4% of measured end-to-end time in denoising, so fewer qualified NFEs, attention/GEMM work, selective quantization and safe cache reuse have the largest ceiling. Turbo4 reduced denoising to 63.6%; after INT8+Sage it fell to 53.8%, making conditioning, VAE and request-pipeline work materially more important. See the measured bottleneck ledger.