SGLang + Claude Code + Qwen 3.8 Flash-Next: A 9-Hour Agentic Session on One DGX Spark

Summary

The video is a hands-on walkthrough of running an open-weight model through SGLang and using it with Claude Code as an agentic coding harness. The creator discusses inference settings, monitoring, a local agent’s tools, and a live coding test.

Setup and performance considerations

• The SGLang configuration uses tensor parallelism of 1, quantization, four speculative tokens, and capacity for four concurrent requests.
• The creator found that the maximum request count and Mamba cache size are closely related memory controls. The available context had to be reduced from roughly 450K to 172K to fit the setup.
• Other discussed options include a reasoning analyzer, a custom tool-call/chat template, and a static-memory setting that the creator says has been stable in testing.
• A precomputed 51-billion-entry n-gram table is offloaded to NVMe because the GPU cannot comfortably hold it alongside the model and KV cache.
• The creator describes an out-of-memory failure caused by a systemd restart race during the model’s multi-phase startup. They recommend relying on Docker’s restart policy instead of adding systemd for this container.
• Prometheus and Grafana are used to expose and inspect inference metrics. The creator also enables cache reporting to track Claude Code’s cached-token usage and per-call telemetry.

Agent features and observability

The local agent, called LangBang in the subtitles, can use shell and file tools, search files, crawl URLs, run subagents, manage scheduled tasks, and perform RAG searches. Its knowledge system includes searchable notes, URLs, local files, PDFs, and Markdown. The creator describes roughly 54 skills, covering areas such as coding, GitHub, media, ML operations, and productivity.

Per-call measurements include context and output tokens, time to first token, prefill speed, decoding speed, and total duration. One example produced about 37 tokens per second, with a later cached interaction reaching about 40 tokens per second.

A trajectory view shows the sequence and duration of model and tool calls. The creator uses it to identify where time is spent, including long waits caused by processes such as video generation or monitoring commands.

Live coding test and evaluation

The creator asks the agent to build a modern, interactive landing page for an algorithmic-trading product, using glassmorphism, mesh gradients, and RAG research.

The agent performs two RAG searches, creates a task list, and begins creating the project and running commands. The task list and trajectory view are highlighted as useful ways to monitor progress.

The creator compares the test with a previous setup that usually completed a similar task in about five minutes. In this run, after about 25 minutes, the task was still underway. The creator’s preliminary assessment is that the model may be slower but more thorough and higher-quality; the subtitles do not show the finished page or a final result.

Although the title refers to a nine-hour agentic session, this excerpt mainly documents setup and the live test rather than detailing that full session.

Guides and useful takeaways

1. Tune request capacity and Mamba cache together when working within tight GPU memory limits.
2. Consider offloading large auxiliary tables to NVMe when they compete with model weights and KV cache.
3. Use Docker restart policies carefully; a systemd restart can interfere with a multi-stage container startup.
4. Add metrics and a trajectory timeline to understand model speed, cache effects, and time spent in tool calls.
5. Use an explicit task list for long-running coding jobs so progress is easier to review.

Main speaker and tools

The video’s creator and narrator, Tyrel Barstow, explains the setup and evaluates the test. The local LangBang agent and Claude Code are shown as tools in the demonstrations.