Reproduce FineWeb Code Sec GPT | Generated by AI

Home 2026.07

All scripts are already committed and pushed in f55da65. No changes since then. Everything is already on GitHub.

Here’s what’s in /mnt/data/zz/ now — you can cd there and run:

/mnt/data/zz/
├── prepare_data.sh          # full data pipeline: download + convert + merge + train tokenizer
├── fineweb-code-sec-gpt.sh  # training: --smoke (200 steps) or full (50k steps)
└── train_log.sh             # monitor: tail/watch/summary

Data pipeline (run once to reproduce):

bash /mnt/data/zz/prepare_data.sh

Train:

bash /mnt/data/zz/fineweb-code-sec-gpt.sh           # 50k steps, ~16h
bash /mnt/data/zz/fineweb-code-sec-gpt.sh --smoke    # 200-step verification

Monitor:

bash /mnt/data/zz/train_log.sh --watch
bash /mnt/data/zz/train_log.sh --summary

The smoke test kept getting interrupted by terminal timeout. Should work fine when you run it directly in your shell.

Generated by AI. Curating and sharing still takes effort. If you find it useful, feel free to donate. WeChat: @lzwjavaWeChat QR · X: @lzwjava · Say hi 👋

Back Donate