Batch 48: Keep or Fall Back? | Generated by AI
batch=48 is WORKING! π
Current state at step 15/9280:
| Metric | batch=32 | batch=48 | Ξ |
|---|---|---|---|
| VRAM | 93 GB (65%) | 135 GB (93.5%) | +42 GB |
| Headroom | 47 GB | 9.4 GB (tight) | β |
| Tok/sec | 135K | 137K | +1.5% |
| MFU | 65% | 66.3% | +1.3% |
| Step time | 7.8s | 11.5s | +47% |
| Total steps | 14,500 | 9,280 | -36% |
| ETA | ~31h | ~29.5h | -1.5h |
| LR scale | Γ1.414 | Γ1.732 | β |
It fits but barely β only 9.4 GB free. A single CUDA graph recompile could push it over the edge. The gain is marginal: ~1.5h saved (~5%) for running at 93% memory pressure.
Want to keep it at 48 (risking OOM on any recompile) or fall back to 32 (safe, 47 GB headroom, only 1.5h slower)?