RunPod Chatbot Setup Summary | Generated by AI

Home 2026.07

Here’s the full summary of what was done this session.

RunPod LibreChat + SGLang Setup — Complete Summary

Server

Services Running

Service Internal Port Status Details
SGLang 30000 ✅ Qwen 2.5 14B AWQ, warmed up with CUDA graphs
MongoDB 27017 ✅ Data at /workspace/mongodb
LibreChat 8889 ✅ Serves through nginx proxy on 8888
nginx 8888 ✅ Proxies 8888 → 8889 with 300s timeout

What Was Done

  1. Inspected the server — found SGLang running on 30000 with Qwen 14B AWQ, LibreChat cloned at /workspace/LibreChat with .env and librechat.yaml already configured to use SGLang.

  2. Freed port 8888 — killed Jupyter Lab that was occupying it.

  3. Started MongoDB — mongod --dbpath /workspace/mongodb --logpath /workspace/mongodb/mongod.log --fork (wasn’t running).

  4. Fixed empty actions.ts — packages/data-provider/src/actions.ts was 0 bytes from the shallow git clone. Restored via git checkout. This had a knock-on effect: the client Vite build failed with missing exports (validateAndParseOpenAPISpec, openapiToFunction).

  5. Rebuilt Node native modules — MongoDB driver v6.20.0’s native BSON addon was incompatible with Node v22.23.1. npm rebuild mongodb fixed the ABI mismatch.

  6. Built the client — npm run build:packages then vite build succeeded. The client dist is at /workspace/LibreChat/client/dist/.

  7. Changed port — .env: PORT=8888 → PORT=8889. Added nginx reverse proxy on port 8888 (/etc/nginx/conf.d/librechat.conf) forwarding to 8889. This ensures:
    • External requests hit nginx on 8888
    • nginx adds proper proxy headers and a 300s timeout
    • LibreChat backend runs on 8889 internally
  8. Started LibreChat — Running on port 8889, served externally via nginx on 8888.

Files Saved to runpod/

File Purpose
start_services.sh One-command restart: MongoDB → check SGLang → rebuild native modules → start LibreChat → verify
build_librechat.sh Full rebuild: restore git files → build packages → Vite client build
start_librechat.md Runbook with current state, service table, verification commands
setup_librechat.sh (pre-existing) Older script, port 3080-based, still there for reference

Key Pitfalls Encountered

  1. Shallow git clone → empty actions.ts → client build fails with missing OpenAPI exports
  2. Node v22 vs mongodb v6 → native addon ABI mismatch → process hangs silently on require("mongodb") with no error message
  3. SSH disconnect kills children → nohup/& from SSH don’t survive disconnection, must use setsid + background terminal
  4. LibreChat DOMAIN_SERVER = 0.0.0.0 → wrong for external access, switched to 127.0.0.1 for nginx backend

Access

Open http://213.173.110.200:8888 in your browser. Register an account, then select Qwen 2.5 14B AWQ in the model picker to chat.

Generated by AI. Curating and sharing still takes effort. If you find it useful, feel free to donate. WeChat: @lzwjavaWeChat QR · X: @lzwjava · Say hi 👋

Back Donate