Ethernet is DEAD?? Mac Studio is 100x FASTER!!

Local AI clustering is back on the table—because Apple’s latest software update slashes the latency bottleneck that previously made multi-Mac setups slower than running models on a single machine. In earlier tests with five Mac Studios, adding more computers cut performance by 91%, and the culprit wasn’t GPU horsepower or memory capacity. It was network latency between machines, which forced AI to use slower “pipeline parallelism” and made more advanced parallel strategies unusable.

The new build uses four fully loaded Mac Studios (512 GB unified memory each, 80 GPU cores each, 8 TB storage each), connected as a cluster with Thunderbolt networking plus Ethernet for discovery. The hardware scale is enormous: 2 TB unified memory total, 32 TB storage total, and 320 GPU cores across the cluster. The key change isn’t just moving to Thunderbolt 5; it’s what Apple enabled in a Tahoe 26.2 update. Apple quietly turned on RDMMA (remote direct memory access) over Thunderbolt ports, bypassing the traditional TCP/IP networking stack that adds overhead and delays.

That shift is dramatic in measured terms: latency drops from roughly 300 microseconds to about 3 microseconds. With that improvement, the cluster can finally use tensor parallelism—where multiple machines split the computation of each model layer and work together—without the communication “chitchat” that previously turned tensor parallelism into a slowdown. The earlier math was brutal: with 160 communication events per token and ~300 microseconds per message, waiting time ballooned to tens of milliseconds per token. After RDMMA, those waits shrink enough to make tensor parallelism practical.

Tests show the performance difference clearly. For a Llama 3.37B FP16 model, the older pipeline approach on the cluster lands around 5 tokens per second, while the tensor-parallel approach with RDMA enabled jumps to about 16 tokens per second (and the smaller-model pipeline case also improves when clustered). The pattern holds as model size increases: clustering doesn’t degrade speed the way it did earlier; instead, it often increases throughput while keeping GPU utilization high.

The real stress test comes with very large models. The setup runs DeepSeek’s “Quinn 3 coder 480B” and then scales to Kimmy K2 (a trillion-parameter “thinking” model), using multiple nodes when a single Mac can’t fit the model. The cluster also runs multiple large models concurrently—loading a trillion-parameter model alongside a 671B model and then adding additional Llama variants—while maintaining workable performance. Some instability appears during beta software operations (including failures that require rebooting and reinitializing the cluster), but the overall result is a functional, responsive local system rather than a fragile science project.

Beyond benchmarks, the cluster is used through real applications like Open WebUI, with coding workflows attempted via tools such as Xcode and OpenCode. The system can become overloaded under heavy “thinking” workloads, but it remains usable enough to demonstrate that clustering can support interactive development.

Bottom line: the cluster’s success hinges on network latency—not raw compute—and Apple’s RDMMA-over-Thunderbolt update is what removes the old ceiling. The experiment suggests local multi-Mac AI can now be fast enough to matter, turning clustering from a curiosity into a practical architecture for running the largest models available locally (at least for those willing to pay for the hardware).