Optimization ledger

Current measured plan

The largest Base40 bottleneck is DiT denoising. Work is prioritized in this order:

  1. remove qualified work with NFE/scheduler/distillation design;
  2. reduce weight and activation bytes with selective quantization or reuse;
  3. accelerate attention and dominant GEMMs with shape-qualified kernels;
  4. overlap transfers and phase work when the memory budget allows;
  5. optimize VAE, conditioning and service lifecycle once Turbo makes them material.

Accepted or exposed

Method Measured speed Quality decision
SageAttention2, Base40 1.283× four blind rounds tied; production path
SageAttention2, Base50 1.313× four blind rounds tied; production path
strict Triton RoPE 1.049× SSIM 0.994831; sample-aligned audio identical
Base40 Pattern3 warmup20 1.182× SSIM 0.990041; explicit experimental option
Base40 → Base24 1.594× fast profile; not lossless, motion-sensitive risk
TP2 on two RTX 3080 cards 1.567× exact parallelism; communication limits scaling
two independent RTX 4090 workers 1.893× observed batch concurrency throughput only

Experimental

Method Signal Why it is not default
AdaLN-BF16 hybrid INT8 1.376× on one 15 s Base40 run SSIM 0.857694; needs cross-scene qualification
Base16 2.278× quality margin is insufficient
SGLang INT8+Sage on one 3080 515.357 s hot Base40 signal cross-executor quality and more scenes remain
W4A4/SVDQuant 1.877× full-linear microbenchmark full residual/scale path has not reached video E2E
selected-layer cache whole-stack cache proves reuse exists per-layer history can be 55–110 GiB; reference KV drifts

Rejected or stopped

Route Measurement Decision
global INT8 + Sage2 1.745× reject: motion −21.99%, audio −4.244 dB
FP8-SGL on RTX 4090 7.315 → 9.320 s/step reject: 27.4% slower and no memory reduction
ConvRot INT8 1.38% slower on 4090; 2.12% slower on 3080 reject on these architectures
keep 24 BF16 blocks resident no gain or slower, ~42.5 GB peak reject
request-local fixed preprocessing cache later steps only 0.283% faster stop: no E2E value
remove audio tokens 0.09% timing noise, changed video reject
cache warmup4 ~1.462× reject: SSIM 0.8431
whole-stack Taylor/scaling cache best residual error still ~20.6% stop before E2E

Fewer steps require a new contract

Dropping NFE alone is not the method. A valid 16/24/32-step route must jointly tune the scheduler, flow shift, solver and—when needed—a trained adapter, then requalify spatial instruction following, motion, audio and final decode. Base24 is the current practical anchor; Base16 shows the speed ceiling but not enough quality headroom. Official 4- and 8-step adapters are evidence that few-step inference is a model-and-sampler contract, not just a smaller integer.

The full 30-route matrix preserves exact workloads and negative results.