Chinchilla Training Script Commands | Generated by AI

Home 2026.06

Checkpoint at step 10,000 already exists. Script is ready:

/mnt/data/nanochat/run_rtx4070_chinchilla.sh

To run fresh from step 0:

bash /mnt/data/nanochat/run_rtx4070_chinchilla.sh

To resume from the existing step 10k checkpoint:

bash /mnt/data/nanochat/run_rtx4070_chinchilla.sh --resume-from-step=10000

What it does:

  87,000 steps x 65,536 batch = 5.7B tokens (Chinchilla optimal)
  Checkpoints every 10k steps (crash-safe)
  Eval every 2k steps, samples every 5k steps
  ~28.5 hours on RTX 4070
  Data: 142.6B tokens on disk, only needs 5.7B

Run it? Or want to tweak anything first (eval frequency, save frequency, etc.)?

Generated by AI. Curating and sharing still takes effort. If you find it useful, feel free to donate. WeChat: @lzwjava · X: @lzwjava · Say hi 👋

Back Donate