Departments that want a private LLM are pricing ex-datacentre GPU servers against new HGX. Here is what DeepSeek V4 Flash and V4 Pro actually need in GPU memory and software support, and where V100, A100, V620, H200 and RTX PRO 6000 land.
This week the practitioner forums keep circling the same question. Research groups handling sensitive data want an LLM that never leaves the building, home and lab builders are filling 4U cases with ex-datacentre accelerators, and four-GPU V100 servers are being offered second hand at prices that look like a bargain. The catch is that the model everyone names, DeepSeek V4, sets a memory and software floor that a lot of that older hardware sits below.
What DeepSeek V4 needs
- DeepSeek V4 comes in two Mixture-of-Experts models. V4 Pro has 1.6 trillion parameters with 49 billion active per token. V4 Flash has 284 billion with 13 billion active. Both support a one million token context.
- The released weights are a mix of FP4 (expert weights) and FP8 (attention, norms and router), so the hardware has to handle those low-precision formats well.
- According to the vLLM serving recipes, the V4 Flash checkpoint is around 160 GB on disk (about 167 GB for the version with the speculative decoding module). V4 Pro is around 865 GB to 893 GB on disk and roughly 960 GB once loaded in mixed precision.
- Weights are only part of it. The KV cache for long contexts and multiple concurrent users needs headroom on top. vLLM caps V4 Pro context on an 8x H200 node because 1,128 GB of total GPU memory leaves the KV cache tight.
| Model | Total / active params | Checkpoint on disk | Hardware the vLLM project lists as verified |
|---|---|---|---|
| DeepSeek V4 Flash | 284B / 13B | approx. 160 to 167 GB | H200, B200, GB200, B300, GB300, DGX Station GB300, DGX Spark GB10, 8x RTX PRO 6000, Instinct MI300X / MI325X / MI355X |
| DeepSeek V4 Pro | 1.6T / 49B | approx. 865 to 893 GB | H200, B200, GB200, B300, GB300, Instinct MI355X (MI300X and MI325X listed as unsupported) |
Where the second-hand GPU servers land
| Platform | GPUs and memory | Total GPU memory | V4 fit |
|---|---|---|---|
| Supermicro SYS-1029GQ-TVRT (1U, dual Xeon Scalable) | 4x Tesla V100 SXM2 32 GB HBM2, 900 GB/s each, NVLink up to 300 GB/s | 128 GB | Below the V4 Flash checkpoint size, and V100 is outside current vLLM support |
| Supermicro AS-2124GQ-NART (2U, dual EPYC 7002/7003) | HGX A100 4-GPU, 40 GB or 80 GB each | 160 GB or 320 GB | Not on either V4 verified list |
| Supermicro AS-4124GO-NART (4U, dual EPYC 7002/7003) | HGX A100 8-GPU 80 GB with NVSwitch, 2,039 GB/s per GPU | 640 GB | Holds Flash comfortably on capacity, but not a verified V4 platform |
| Radeon PRO V620 (PCIe Gen4, passive, dual slot) | 32 GB GDDR6, 512 GB/s, 300 W board power | 32 GB per card | Not a vLLM ROCm target |
| 8x NVIDIA RTX PRO 6000 | 96 GB per GPU | 768 GB | Verified for V4 Flash |
| HGX H200 8-GPU | 141 GB per GPU | 1,128 GB | Verified for Flash and Pro (Pro with a capped context) |
- Software support is the hidden cut-off. Current vLLM documentation lists NVIDIA compute capability 7.5 or higher. The Tesla V100 is compute capability 7.0, and vLLM's own legacy V100 tutorial notes that newer prebuilt images dropped it, so a V100 box means pinning old software.
- The A100 runs current vLLM, but its tensor cores have no native FP8 or FP4, which is how V4 is shipped. It remains a strong platform for smaller dense and quantised models, just not the default home for V4.
- The V620 is an RDNA2 card (gfx1030). It is in AMD's ROCm support tables, but vLLM's ROCm build targets Instinct MI200, MI300 and MI350 plus RDNA3 and RDNA4 Radeon cards. Being passively cooled, it also needs proper server airflow, which is why builders end up moving them into 4U chassis.
Rule of thumb for V4: add up total GPU memory, subtract the checkpoint size, and what is left is your KV cache for context and concurrent users. If the answer is negative or small, the model does not fit, however good the price.
Picking a platform for a department
- One team, V4 Flash, sensitive data: an 8x RTX PRO 6000 PCIe server is the verified option that avoids an HGX baseboard. DGX Station GB300 is the deskside route if there is no rack space.
- Many users or V4 Pro: an 8-GPU HGX H200 or Blackwell system is the floor the recipes are written for.
- Existing A100 HGX estate: keep it for fine-tuning, embeddings, retrieval and smaller models, and add a verified node for V4 rather than forcing it onto Ampere.
- V100 and V620 boxes: useful for learning, CUDA or ROCm development and small models. Budget them as lab kit, not as the production endpoint.
The ETON view
The forum conversation is right about one thing: a department that wants a private model should look at second-hand GPU servers, because that is where the value is. But price per GPU is the wrong metric. Price per usable gigabyte on a supported software stack is the right one, and on that basis a cheap four-way V100 box is lab kit, while an A100 HGX is a capable platform for everything except the newest FP8 and FP4 models.
For V4 specifically, we would scope Flash on an 8x RTX PRO 6000 PCIe server or an HGX H200, and Pro on HGX H200 or Blackwell. The PCIe route matters in the UK because many colocation racks are still provisioned at 6 to 12 kW, and an air-cooled PCIe server is easier to place than a liquid-ready HGX system. HGX H200 and Blackwell allocation still drives lead times, so quote the GPUs separately and lock them first.
We are vendor neutral across Supermicro, Dell, HPE, GIGABYTE and ASUS GPU platforms, new and refurbished, and we buy back retired V100 and A100 systems. That buyback value is often what makes the step up to a verified platform affordable. Tell us the model, the number of users and your rack power and we will tell you what fits, what we hold, and what the lead time is.
Related infrastructure
Category: AI & GPU · Vendor: NVIDIA, AMD, Supermicro · Technology: DeepSeek V4, HGX A100, HGX H200, Tesla V100 SXM2, RTX PRO 6000, Radeon PRO V620 · Last verified: 07 Oct 2026 · ~5 min read
Sourcing this kind of infrastructure? Talk to ETON about availability, lead time and pricing.
