Flux CPU Generation Bottleneck

Flux CPU Generation Bottleneck | Generated by AI

Home 2026.07

Question: Explain the final generation stage and performance bottleneck in this Flux CPU run

This part is the VAE decode + cleanup + save phase. The important line:

generate_image completed in 526.73s

means:

one 1024×1024 FLUX.1-schnell image took ~8 minutes 47 seconds on CPU.

The model loading was not the problem. The compute was.

1. VAE backend buffer

model manager prepared params backend buffer
(94.57 MB, 138 tensors, RAM)

This is the VAE model.

Pipeline now:

Flux Transformer
        |
        v
latent image
        |
        v
VAE decoder
        |
        v
RGB pixels

VAE is small:

94.57 MB weights

Compared with Flux:

Flux transformer:
6389 MB

VAE:
95 MB

2. VAE compute buffer

vae compute buffer size: 6657.00 MB(RAM)

This is the interesting part.

The VAE weights are tiny, but the temporary activation memory is huge.

Similar to neural network inference:

weights
+
activations
+
workspace

For VAE:

weights:
95 MB

workspace:
6657 MB

Why?

Because decoding:

latent
  |
  | upsampling
  |
  v
high resolution feature maps
  |
  v
1024x1024 RGB

creates large intermediate tensors.

3. VAE decode time

computing vae decode graph completed, taking 37.20s

So:

VAE:
37 seconds

Total:

526 seconds

Only:

37 / 526

≈ 7%

of total time.

The Flux transformer is the real bottleneck.

4. Total generation breakdown

Approximate:

Text encoding:
2.2s

Flux transformer:
~487s

VAE:
37s

----------------
Total:
526s

Architecture:

Prompt
 |
 | 2 sec
 v
CLIP
 |
 v

Flux Transformer Q4
 |
 | ~8 min
 v

latent

 |
 | 37 sec
 v

VAE

 |
 v

PNG

5. Why Flux is slow on CPU

The Flux model:

12B parameters
hidden size
attention heads
transformer blocks

Even quantized:

6.4GB weights

but every token needs transformer computation.

1024×1024:

image_seq_len = 16384

Meaning attention operates over:

16384 tokens

Very similar to running a long-context LLM.

Example:

Llama:

8192 text tokens

Flux:

16384 image tokens

The compute is enormous.

6. Why GPU would change everything

Your RTX 4070:

CUDA cores:
5888

VRAM:
12GB

Your model:

Flux Q4:
6.4GB

compute:
2.3GB

VAE:
~1GB

total:
<10GB

Fits.

Expected difference:

CPU:

526 sec

GPU:

probably:

20-60 sec

depending on CUDA backend.

7. Current run is basically “llama.cpp for images”

This setup is analogous to:

llama.cpp
+
7B Q4 model
+
CPU inference

but for image generation.

The GGUF line:

flux1-schnell-Q4_0.gguf

means:

LLM quantization ideas
        |
        v
image diffusion models

Same tradeoff:

FP16:
quality + speed + huge memory

Q4:
small memory + slower

8. The next optimization path

For your RTX 4070, I would not run this CPU mode.

Check build:

./stable-diffusion --version

or:

ldd ./stable-diffusion | grep cuda

You want:

GGML_CUDA=ON

Build:

git clone https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp

cmake -B build \
  -DSD_CUDA=ON

cmake --build build --config Release -j

Then run:

./build/bin/sd-cli \
  -m flux1-schnell-Q4_0.gguf \
  --cfg-scale 1 \
  --steps 4

You should see:

VRAM:
~9000 MB

instead of:

VRAM:
0 MB
RAM:
6784 MB

The log is actually a nice demonstration: Flux Q4 has crossed the point where a 12GB consumer GPU can run a 12B image model locally. The remaining issue is not model size anymore; it is getting the compute onto CUDA.

Back Donate