vLLM Serve Command on Radeon Cloud

vLLM Serve Command on Radeon Cloud | Generated by AI

Home 2026.07

Question: How to fill the vLLM “Serve Command” field in Radeon Cloud (AMD OneClick Base ROCm 7.2.1)?

Answer:

The Serve Command is simply the vllm serve CLI command that starts the OpenAI-compatible API server. vLLM syntax is:

vllm serve <HuggingFace-model-id> [options]

(vLLM)

For example, if you want to deploy Qwen3 8B:

vllm serve Qwen/Qwen3-8B \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype float16 \
  --max-model-len 32768

Put that whole command into Serve Command.

Common examples:

1. Qwen3 8B (good starting point)

vllm serve Qwen/Qwen3-8B \
--host 0.0.0.0 \
--port 8000 \
--dtype float16 \
--max-model-len 32768

2. Qwen3 14B

vllm serve Qwen/Qwen3-14B \
--host 0.0.0.0 \
--port 8000 \
--dtype float16 \
--max-model-len 32768

3. Larger model with multiple AMD GPUs

Example: 4 GPUs:

vllm serve Qwen/Qwen3-32B \
--host 0.0.0.0 \
--port 8000 \
--dtype float16 \
--tensor-parallel-size 4 \
--max-model-len 32768

--tensor-parallel-size splits the model across GPUs. (Mintlify)

4. For long-context agent usage

Since you use agents (Claude Code/OpenClaw style workloads), KV cache becomes the bottleneck. Start with:

vllm serve Qwen/Qwen3-30B-A3B \
--host 0.0.0.0 \
--port 8000 \
--dtype float16 \
--max-model-len 65536 \
--gpu-memory-utilization 0.95

Important for Radeon ROCm

The container:

amd-oneclick-base:rocm7.2.1-py3.12-v20260416

already has ROCm + vLLM environment prepared. AMD documents vLLM serving through ROCm with OpenAI-compatible APIs. (ROCm Documentation)

You usually do not need:

python -m vllm.entrypoints.openai.api_server

because vllm serve is the current interface. (vLLM)

A good first test on Radeon Cloud:

vllm serve Qwen/Qwen3-8B --host 0.0.0.0 --port 8000 --dtype float16

After deployment you should get:

https://<space-id>/8000/v1

Then test:

curl https://<space-id>/8000/v1/models

or with Python:

from openai import OpenAI

client = OpenAI(
    base_url="https://<space-id>/8000/v1",
    api_key="<API_KEY>"
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[
        {"role": "user", "content": "hello"}
    ]
)

print(response.choices[0].message.content)

References:

Back Donate