By Adri 
Local artificial intelligence is moving beyond the idea of running one small model on a single computer.
Developers are increasingly exploring ways to connect compact systems and distribute demanding workloads across several accelerators.
NVIDIA has published a detailed demonstration of that approach.
In a technical article dated October 22, four NVIDIA engineers describe how inference, model orchestration and training scale across as many as eight DGX Spark systems.
The computers are designed to support AI development on locally controlled hardware without requiring every request to be processed by an external cloud provider.
The significance of the experiment goes beyond the total amount of memory available.
NVIDIA also examines how interconnect topology, communication algorithms and workload organization affect practical performance.
The results could interest researchers and development teams that want local control over models and data.
However, these are vendor-produced benchmarks for selected workloads, not independent performance guarantees.
Eight compact systems provide approximately one terabyte of memory
Each DGX Spark used in NVIDIA's experiments includes 128GB of unified memory.
That architecture allows the processor and GPU to work with a shared memory space rather than relying solely on the dedicated graphics memory found in many conventional systems.
One DGX Spark provides 128GB.
Four systems provide a combined 512GB, while eight reach 1,024GB, approximately one terabyte in decimal units.
This capacity matters for large language models.
A model's weights require memory, alongside temporary computation buffers and the information generated while processing requests.
Models containing hundreds of billions of parameters can exceed the capacity of individual workstations.
Distributing their weights across multiple systems provides a way to overcome that limit.
However, combining the memory capacity of eight computers does not make them physically equivalent to a single enormous GPU.
Information remains distributed across devices.
Those devices must communicate during computation, creating a dependency on the performance of the interconnect.
The full 753-billion-parameter GLM 5.3 model runs locally
NVIDIA used GLM 5.3 from Z.ai to demonstrate the memory capacity of the cluster.
The full model contains 753 billion parameters.
Its NVFP4-quantized checkpoint occupies approximately 470GB, according to the company's technical report.
Quantization reduces numerical precision to lower memory requirements and can improve the efficiency of certain operations.
It also introduces potential tradeoffs depending on the model and quantization method.
A checkpoint occupying 470GB does not mean any system with exactly 470GB of memory can necessarily serve it effectively.
Additional memory is required for inference software, intermediate results and request processing.
Across eight DGX Spark systems, NVIDIA measured approximately 31 generated tokens per second with a single request.
The combined throughput increased to 51 tokens per second with four concurrent requests.
At eight concurrent requests, aggregate generation reached 67 tokens per second.
However, performance per user declined as concurrency increased.
For eight simultaneous requests, NVIDIA reported approximately 18 tokens per second per user under the published conditions.
This illustrates the difference between total server throughput and the speed experienced by an individual user.
A system can process more work overall while each individual request receives fewer resources.
GLM 5.3 Flash reaches approximately 80 tokens per second
NVIDIA also tested GLM 5.3 Flash.
That variant reached approximately 80 tokens per second in the published demonstration.
This is notable for a local system running a large mixture-of-experts model.
However, it should not be confused with the performance of the full GLM 5.3 checkpoint.
Different model variants can have different architectures and inference characteristics.
It would therefore be inaccurate to claim that the full 753-billion-parameter model consistently delivers 80 tokens per second.
Performance is also sensitive to context length, concurrency and inference optimizations.
The results establish that very large models can operate on the demonstrated local configuration.
They do not establish universal performance or cost advantages over cloud infrastructure.
Eight machines are not eight times faster
NVIDIA's Qwen3.8-27B benchmark provides a clearer view of distributed scaling.
The company measured inference performance across one, two, four and eight DGX Spark systems using different network arrangements.
A single machine delivered 11.86 generated tokens per second in the selected test.
Two directly connected machines reached 21.71 tokens per second.
Four systems connected in a physical ring achieved 32.82 tokens per second.
Finally, eight systems connected through a network switch and using a specialized optimization reached 52.58 tokens per second.
That represents a 4.43-fold speedup compared with one machine.
It is a substantial improvement, but well below the theoretical eightfold increase associated with multiplying the hardware count by eight.
The difference is partly caused by the communications required to synchronize work across accelerators.
When model computation is distributed across machines, certain operations depend on results from other devices.
Networking then becomes part of the computational cost.
This is where NVIDIA's new communication optimization becomes relevant.
RoCEnante accelerates communication between accelerators
NVIDIA describes a specialized collective communication mechanism called RoCEnante.
It targets operations such as all-reduce, where multiple processing devices need to combine their results.
Traditional ring-based algorithms exchange information progressively between neighboring devices.
That approach can be efficient, but some workloads incur latency from the multiple communication steps involved.
RoCEnante uses a different method for selected small clusters.
Each participating machine exchanges data directly with the others and performs the final reduction locally.
This can reduce communication latency when messages are relatively small and the number of devices is limited.
In the eight-node Qwen3.8-27B test, NVIDIA measured a 4.43-fold speedup using RoCEnante.
Without the optimization, the same general switched configuration achieved a 3.05-fold speedup.
NVIDIA describes the difference as an approximately 45% improvement in scaling efficiency.
This should not be interpreted as a universal 45% speed increase for all DGX Spark applications.
The improvement was measured for a particular model, topology and benchmark configuration.
RoCEnante does not replace NCCL
The communication method also has limitations.
Directly exchanging data with every other participant increases the amount of network traffic.
As the number of nodes grows, that overhead can become significant.
Conventional ring-based methods may remain more efficient for larger messages or bigger clusters.
NVIDIA explicitly says RoCEnante is not a replacement for its NCCL collective communication library.
The specialized backend is designed primarily for smaller switch-connected systems with communication patterns that benefit from lower latency.
It has been integrated into FlashInfer, which can fall back to NCCL when the optimization's requirements are not satisfied.
This distinction matters to developers.
There is no single networking algorithm that is optimal for every AI workload.
The right approach depends on model architecture, communication volume, cluster size and concurrency requirements.
Ring networks and switched networks serve different needs
NVIDIA also examines how physical topology affects cluster behavior.
For a simple four-node system running one distributed model, a ring can be attractive.
It avoids the need for a separate network switch and provides useful performance under the tested conditions.
However, more complex deployments may need direct communication between different groups of machines.
A development team might dedicate four devices to one large model while using the remaining systems for independent agents.
A switched network provides greater flexibility for that arrangement.
RoCEnante also requires the direct point-to-point connectivity associated with switched configurations.
DGX Spark systems use NVIDIA ConnectX-7 networking and RDMA over Converged Ethernet to exchange information efficiently.
Interconnect performance becomes especially important when accelerators need frequent synchronization.
The configuration is therefore more sophisticated than simply connecting several compact computers to an ordinary home network.
Distributed inference requires appropriate networking and software that understands how to divide computation across devices.
Multiple smaller models can outperform a large-model-only approach
The next experiment focuses on agent orchestration.
NVIDIA used NeMo Switchyard to distribute requests between different model pools.
The idea is straightforward: not every task requires the largest available model.
A smaller model may be sufficient for routine questions, while more demanding requests can be routed to a stronger model.
In the published prototype, MiniCPM5 2B handles simpler requests while Qwen3.8 Flash Next processes more difficult ones.
A lightweight classifier determines which model should receive each request.
The eight DGX Spark systems are divided into groups running different model replicas.
NVIDIA describes a configuration with three Qwen replicas using two DGX Sparks each and two MiniCPM replicas using one Spark each.
The arrangement allows different requests to be handled in parallel.
On a 92-case benchmark involving tool use, the routed configuration received a reported score of 91 out of 100, matching the Qwen-only reference system.
When 75% of requests were routed to the smaller model, NVIDIA reported approximately three times the aggregate throughput.
This suggests that intelligently distributing work can be more valuable than sending every request to the strongest model.
However, the quality of the routing decision is critical.
A difficult task sent to an insufficiently capable model may produce a worse result.
The threefold improvement is therefore workload-specific rather than a guaranteed outcome for all agent systems.
Distributed training shows nearly linear scaling
NVIDIA also investigated training rather than inference.
The company pretrained a nanochat model containing approximately 896.5 million parameters across one to eight devices.
In this setup, each node maintains a full copy of the model and processes training data.
The systems synchronize gradients during training steps.
This differs from distributing the weights of a much larger model for inference.
The results showed strong scaling.
A single DGX Spark processed 8,741 tokens per second.
Two devices reached 17,202 tokens per second.
Four devices achieved 33,640 tokens per second.
Eight systems delivered 68,079 tokens per second.
The eight-node configuration was 7.79 times faster than a single device, corresponding to 97.4% scaling efficiency in NVIDIA's measurements.
That is close to the ideal performance increase from using eight machines.
However, the result concerns a relatively small model and a particular distributed training configuration.
It should not be generalized to training models with hundreds of billions of parameters.
Their memory requirements and communication patterns can be very different.
Local inference is not automatically cheaper than cloud computing
Running advanced models on locally managed hardware offers potential benefits.
Development teams can keep data within their own environment, test different software configurations and reduce dependence on externally hosted inference services.
They can also access a known amount of computing capacity without paying a cloud provider separately for every locally generated token.
However, owning infrastructure introduces other costs.
Those include hardware purchases, networking, storage, electricity, maintenance and engineering time.
An eight-node cluster also requires a substantial upfront investment.
For organizations that only occasionally need very large models, rented cloud capacity may remain more economical.
Teams with steady workloads and specific data-control requirements may see a stronger argument for local systems.
The economics depend on utilization, workload characteristics, required throughput and the prices of alternative services.
NVIDIA's October 22 technical article does not provide a comprehensive total-cost-of-ownership comparison against equivalent cloud or data-center infrastructure.
It would therefore be premature to claim that eight DGX Spark systems are always the cheaper option.
Eight-node automation in NVIDIA Sync is still coming
The benchmarks should not be confused with complete commercial availability of every management feature.
NVIDIA says DGX Spark clusters of up to eight nodes are supported through manual configuration.
The published results demonstrate that the hardware and software can operate together under the tested conditions.
However, full eight-node cluster support in NVIDIA Sync Cluster Assistant is planned for the end of 2026.
The assistant is designed to simplify configuration, networking and device management.
NVIDIA is also preparing Model Launcher to help developers download and start local AI models.
These software milestones must be distinguished from the hardware demonstrations.
The eight-node experiments have been performed.
But the fully simplified management experience promised for future NVIDIA Sync releases is not yet available across every configuration discussed.
A new 64GB DGX Spark configuration arrives separately
NVIDIA is also expanding the DGX Spark product lineup.
The company announced a 64GB unified-memory configuration scheduled to become available through hardware partners on October 23.
US pricing starts at $4,999, according to NVIDIA's announcement.
The system retains the Grace Blackwell GB10 platform and NVIDIA's AI software stack.
It targets workloads that need less memory than the original 128GB configuration.
However, this smaller-memory product should not be confused with the machines used in the October 22 benchmarks.
NVIDIA's one-terabyte cluster experiments used eight 128GB systems.
Eight 64GB devices would provide a combined 512GB of memory rather than 1,024GB.
The resulting configuration would not be identical to the system demonstrated with full GLM 5.3.
Hardware specifications matter directly when assessing model compatibility and performance.
What NVIDIA's benchmarks actually establish
The primary achievement is demonstrating multiple forms of distributed AI computation on compact hardware.
Eight 128GB DGX Spark systems can load the complete 753-billion-parameter GLM 5.3 model.
NVIDIA measured up to 67 generated tokens per second in aggregate with eight concurrent requests on that model.
With Qwen3.8-27B, an optimized configuration reached 52.58 tokens per second, 4.43 times the throughput of a single Spark in the selected test.
For pretraining a much smaller nanochat model, performance increased by a factor of 7.79.
These results are consistent with each other.
They show that workload characteristics strongly affect how efficiently a cluster uses additional machines.
Training with limited synchronization requirements can scale almost linearly.
Inference that requires frequent communication between devices may gain less from additional hardware.
The important engineering question is therefore not simply how many DGX Sparks can be connected.
It is how models, data and computation should be distributed across the available resources.
A meaningful step for local AI, not the end of cloud infrastructure
NVIDIA's October 22 publication represents an important development for local artificial intelligence.
The category is expanding beyond small models on isolated personal computers.
Research and development teams can increasingly consider compact shared systems capable of loading much larger models.
That could support specialized applications, experiments involving internal data and the development of local agents.
However, compact clusters are not universal replacements for large data centers.
Industrial infrastructure may provide more powerful accelerators, specialized interconnects and architectures better suited to high-throughput production.
NVIDIA's benchmarks should therefore be interpreted as evidence of a technical capability, not proof that desktop hardware beats every cloud alternative.
The future of these systems will depend on usability, reliability and total operating cost as much as benchmark results.
The central lesson is that expanding local AI requires more than adding memory and processors.
It requires making those resources work together efficiently.