Separate Terminals for Training and Analysis

Separate Terminals for Training and Analysis | Generated by AI

Home 2026.05

Question: When using Claude Code alongside a project like lzwjava/zz for ML model training, is the best practice to run long-running tasks (training, tokenizing) in a separate terminal, while using Claude Code in another terminal to analyze logs and code to evaluate training progress?

Answer:

Yes, you are exactly right. This is the recommended and practical workflow pattern. Here is a comprehensive breakdown:

The Two-Terminal Workflow for ML Training with Claude Code

Why You Should NOT Run Training Inside Claude Code

Long-running ML tasks — training, tokenization, dataset downloads — can take hours. Running them directly inside a Claude Code session is problematic because:

Claude Code sessions can time out or lose context over long durations
You tie up your Claude Code session waiting on a blocking process
Claude cannot observe live stdout/stderr the same way you can in a real terminal
Claude performs dramatically better when it can verify its own work — without a clear feedback loop, you become the only error-checker, and every mistake requires your attention

Terminal 1 — The “Worker” Terminal (No Claude Code)

This terminal is for heavy, long-running processes. You run them directly:

# Tokenize datasets
python scripts/extract/extract_fineweb.py

# Run training (hours-long)
python train.py --config config.yaml 2>&1 | tee logs/train_log_openweb.txt

# Download datasets
bash scripts/download/wget_fineweb_1.sh

Key rules:

Redirect output to log files with tee so logs persist
Use nohup or tmux/screen to keep processes alive if the terminal closes
Let it run unattended — do not babysit it

Terminal 2 — The “Claude Code” Terminal (Analysis & Code Queries)

This terminal is where Claude Code lives. While training runs in Terminal 1, you use Claude Code here to:

Read and analyze training logs — point Claude at your logs/train_log_openweb.txt file
Query the codebase — ask Claude to explain what a training script does, how loss is computed, etc.
Evaluate results — use scripts like python scripts/analysis/evaluate.py --file logs/train_log_openweb.txt
Iterate on code — ask Claude to suggest improvements to training config, learning rate schedule, tokenizer, etc.

Example prompts you’d give Claude Code in Terminal 2:

@logs/train_log_openweb.txt The loss seems to plateau after step 3000. 
Why might this happen and how should I adjust the learning rate?

@scripts/analysis/evaluate.py Analyze this evaluation script — 
what metrics is it computing and are there any bugs?

Recommended Enhancements to This Workflow

Use tmux to manage both terminals cleanly:

tmux new-session -s training
# Pane 1: training job
# Pane 2: claude (Claude Code session)

The Claude Code team recommends keeping a dedicated “analysis” worktree just for reading logs and running queries — many engineers run 3–5 Claude sessions in parallel, each in its own git worktree.

Always log training output to files:

python train.py 2>&1 | tee logs/train_$(date +%Y%m%d_%H%M%S).txt

This way Claude Code can read the full log history, not just what’s currently on screen.

Use Claude Code’s analysis scripts (like those in lzwjava/zz):

The repo already has scripts/analysis/calculate_duration.py and scripts/analysis/evaluate.py — these are exactly the right kind of scripts to run from Terminal 2 and have Claude interpret the results.

A good practice is to always ask Claude to run the terminal commands you want to observe as background tasks — this gives better visibility into logs for debugging.

Summary Table

	Terminal 1 (Worker)	Terminal 2 (Claude Code)
Purpose	Execute long jobs	Analyze & iterate
What runs	`train.py`, `tokenize.py`, `download.sh`	`claude` CLI
Duration	Hours	Interactive
Output	Log files	Insights, code edits
Claude involved?	No	Yes

This separation keeps your expensive GPU/CPU time fully utilized, while letting Claude Code act as your intelligent analyst and code assistant on the side — exactly as you described.

References:

Back Donate