Homelab

Two NVIDIA DGX Spark nodes on my desk, linked over NVIDIA ConnectX-7 200 GbE networking and running local LLM inference 24/7.

Why I built it

A home AI cluster I fully control: always-on, private inference for my own agents and experiments — a local alternative to renting cloud GPUs. The real draw is getting to run frontier-class open-weight models across more than one GPU node: sharded serving, speculative decoding, and A/B testing engines (vLLM vs SGLang) on the same hardware.

The hardware

	Node 1	Node 2
Unit	NVIDIA DGX Spark	NVIDIA DGX Spark
SoC	GB10 Grace Blackwell	GB10 Grace Blackwell
Unified memory	128GB	128GB
OS	Ubuntu + Docker	Ubuntu + Docker

Each GB10 chip pairs the 20-core Arm CPU and GPU over NVLink-C2C inside one package, with 128GB of co-packaged unified memory. Between the two systems, fast networking comes from NVIDIA ConnectX-7 200 GbE interfaces that carry RDMA — that low-latency interconnect is what makes multi-node tensor parallelism practical.

Topology

  ┌────────────────────────┐   ConnectX-7 200 GbE   ┌────────────────────────┐
  │   DGX Spark — Node 1   │◄───────────────────────►│   DGX Spark — Node 2   │
  │   gx10-node-1          │   10.100.144.x (RDMA)   │   gx10-node-2          │
  │   GB10 · 128GB         │                             │   GB10 · 128GB         │
  │   vLLM / SGLang        │                             │   vLLM / SGLang        │
  └───────────┬────────────┘                             └────────────┬───────────┘
              │   Docker · Tailscale                                │
              └────────────────── Router / WiFi ────────────────────┘

The stack

• Docker — everything runs as containers on both nodes, one swarm-like orchestration layer on top.
• vLLM — primary serving engine, tuned with NVFP4 KV-cache, prefix caching, chunked prefill, and FlashInfer autotune.
• SGLang — alternative engine used for vision builds and higher-concurrency experiments.
• VLLMController — the control plane I built around all of this: reusable recipes, one-click deployments, launch controls, and MCP-driven automation, so agents can spin models up and down.
• Tailscale — reach the cluster from anywhere, including my phone.

What it runs

• DeepSeek-V4-Flash — served sharded across both nodes (tensor parallel 2) with DSPARK speculative decoding, an NVFP4 quantized build, and roughly a 400k-token context window.
• Vision variants — BF16 and NVFP4 multimodal builds of the same model family, run through both vLLM (NVFP4 MLA path) and SGLang (DSPARK) to compare throughput and stability.
• Speculative decoding — the DSPARK draft model meaningfully cuts time-to-first-token and steady-state latency at low batch sizes.

Why multi-node

Most homelab setups split one GPU’s worth of memory across a model. Here the two Sparks behave like a small distributed cluster: a 400k-context serving stack with speculative decoding fits comfortably, and I can benchmark engine variants over the same mid-plane. When I want, I can also run two independent models side-by-side instead.

Tech stack

• Hardware: NVIDIA DGX Spark ×2 (GB10, 128GB unified memory each)
• Interconnect: NVIDIA ConnectX-7 200 GbE (RDMA)
• Serving: vLLM, SGLang
• Models: DeepSeek-V4-Flash (BF16 + NVFP4, vision variants)
• Orchestration: VLLMController (recipes, deployments, MCP)
• Ops: Docker, Tailscale

Status

Live at home 24/7. The model catalog and recipe set keep growing; next up on the list is stable self-hosted vision and exposing everything to my own agents over the local network.