NVIDIA Runs 753-Billion-Parameter GLM-5.3 Across Two DGX Stations

An NVIDIA demonstration ran the 753-billion-parameter GLM-5.3 NVFP4 model across two connected DGX Stations, showing how local inference can serve a model whose weights exceed one GB300 GPU’s memory.

The model needs about 464 GB of GPU memory for its weights, compared with 252 GB of HBM3e memory on a single GB300 GPU. GLM-5.3 has 753 billion parameters in total and activates 40 billion for each token it generates. The model uses a mixture-of-experts design, so the full set of weights must remain available as tokens are routed to different experts.

NVIDIA linked two GB300 GPUs using ConnectX-8 RDMA and split the model’s tensors and computation between them with tensor parallelism. Station A runs an OpenAI-compatible API endpoint, while Station B works as a worker. Both systems process each request, and Station A returns the completed response.

The deployment used vLLM with prefix caching, an FP8 key-value (KV) cache and multi-token prediction. On the SPEED-Bench Coding dataset, throughput reached 607.8 tokens per second with 16 requests running at once. At 32 concurrent requests, the system delivered 385 tokens per second, and mean request latency was 453.5 seconds. Concurrency refers to the number of requests being handled at the same time.

The setup called for two DGX Stations running Linux and a network connection validated with NVIDIA’s Connect Two DGX Stations playbook. Requirements also included Docker, the NVIDIA Container Toolkit, and access to the dataset and licenses. Each station also required 464 GB of available storage for the model checkpoint, plus space for the vLLM container and temporary files.

This is a different configuration from TokenPost’s earlier coverage of an eight-system DGX Spark cluster with up to one terabyte of unified memory. The current demonstration measures a model split across two DGX Stations.