INFINITE Inference Power for AI

A single Camino Grand server built around six NVIDIA 4090 GPUs is positioned as a cost-effective “inference powerhouse,” delivering far more usable large-model throughput than buying equivalent compute from top-tier datacenter accelerators—while staying within a roughly low–$30,000 price range. The pitch is straightforward: for workloads dominated by inference (running models to generate answers), this setup can deliver “bang for your buck” that undercuts the cost of a single H100, making it attractive for teams that need lots of model calls rather than heavy training.

The hardware design is central to the claim. Fitting six 4090s into one chassis is made possible by treating the cards as consumer GPUs that are still cooled aggressively enough for continuous operation. Camino ships the system fully water-cooled, and power delivery is handled by four power supplies totaling about 6 kW. The transcript also contrasts this with earlier Camino testing: server GPUs are built to tolerate high temperatures for long durations, but electrical heat still degrades components over time—so the cooling strategy matters. In a sustained stress test running all six 4090s at full utilization for about an hour, temperatures stabilized in the upper 60s to low 70s °C with ambient around 24 °C, while noise rose sharply at maximum performance (about 77 dB from a meter).

Why use 4090s in a server at all? The argument is price-to-performance. The 4090 is described as “S tier” for cost efficiency, with the main shortcomings tied to enterprise training features: no NVLink and no SXM variant, which limits memory pooling across GPUs. That makes the system less suitable for training models that require spanning multiple GPUs, where alternatives like A100, H100, or larger-memory datacenter GPUs (the transcript mentions a 144 GB-per-card G200) are still preferred.

Where the Camino Grand shines is inference. The tester runs the open-source “Qwen 72B,” described as approaching GPT-4-like performance in some respects. In practice, Qwen 72B is characterized as strong at factual understanding, reasoning, and programming, but weaker at strict instruction-following and prompt-structured behavior. Beyond chat, the system is used for an “educational lecturer” style project where the model streams topic content and can be directed asynchronously, with ideas to improve chunked streaming, delivery speed, and voice options.

The machine also supports real-time robotics experiments via image-to-depth inference. A Wi‑Fi-based quadcopter (a small “Rise T” drone) sends camera frames to the Camino server, where an RGB-to-depth model generates depth information fast enough to support real-time control loops; running such a depth model on the drone hardware itself is considered impractical due to size constraints. The transcript frames this as a sign that modern open-source models have become “plug and play” compared with earlier years, when tasks like object detection and RGB-to-depth/segmentation were far harder.

Finally, the server is used to probe language-model behavior through multi-model “conversation” experiments. Six models are loaded across six GPUs and prompted with a question plus conversation history, including a philosophical debate about whether AI should have rights. The results are mixed: models can argue on either side, but they often avoid strong opinions and show difficulty engaging in genuine back-and-forth debate. The takeaway is less about hardware alone and more about what compute enables—rapid experimentation with model behavior, robotics perception, and inference-heavy applications—paired with clear limitations around training-scale features and the system’s loud, server-grade operation.