![\"\"](\"https:\/\/www.nextplatform.com\/wp-content\/uploads\/2025\/05\/supermicro-xai-rack-logo-824x438.jpg\" "\\"supermicro-xai-rack-logo\\"\/"){kind=logo}
\n<\/figure>\n
Here is a question for you. What is harder to get right now: 1,665 of Nvidia\u2019s \u201cBlackwell\u201d B200 GPU compute engines or 10 megawatts of power for a four year contract in the Northeast region of the United States?<\/p>\n
Without question, it is the latter, not the former, and both will cost on the order of $66 million.<\/p>\n
The fun bit is that those GPUs will probably actually take 13.4 megawatts of juice to operate in a GB200 NVL72 rackscale system configuration, and that means they will burn around 88.5 megawatts over four years. And if you don\u2019t need a rackscale coherent memory domain for the GPUs because you are using the GPU machinery for AI training instead of inference (which operates at a scale of tens of thousands of GPUs), you will burn about the same power but you can do it with twice as much space and half the power density.<\/p>\n
Here is another fun bit about modern AI datacenters: If you can\u2019t prove that you have the power allocated to you, and in a datacenter that is designed to handle the density of the system, Nvidia will not sell the GPUs to you until you can prove you have power. And, the world on the street last week when we spoke at a conference at the NASDAQ exchange in New York City focused on AI in the financial services industry was that power companies are now trying to stretch their gigawatts of power generation and are increasingly looking at how you are distributing power and doing the cooling in an AI datacenter before they do their allocations.<\/p>\n
Increasingly, if you can\u2019t prove you are using the power wisely, you don\u2019t get it, or you don\u2019t get as much as you want.<\/p>\n
Add to all of this the fact that compute density is necessary in an AI system running chain of thought models because these require coherent memory links between GPUs with super-low latency for AI inference, and we are in a situation where direct liquid cooling is not inevitable in the future, but is absolutely necessary right now. And a lot of datacenters are not used to it, and those that were way back in the IBM System\/360 and System\/370 mainframe days five and six decades ago have not had liquid cooled iron in their datacenters for a long time.<\/p>\n
Which is why companies like Supermicro have to push the envelope on direct liquid cooling for their GPU-accelerated systems.<\/p>\n
\u201cAll of the customers that we talk to are thinking in terms of how many GPUs can they power and cool per megawatt,\u201d Michael McNerney, senior vice president of marketing and network security at Supermicro, tells The Next Platform<\/em>. \u201cThey tell us how many megawatts, and they want the maximum number of GPUs possible. The conversations are about GPU density and GPUs per megawatt, and it is not about how much money they can save on power but getting more GPUs to throw at the AI workload.\u201d<\/p>\n Supermicro developed its first generation of direct liquid cooling with cold plates on the CPUs and GPUs with eight-GPU servers based on Nvidia\u2019s \u201cHopper\u201d H100 GPUs in the fall of 2023, which it first became apparent that some of the cooling techniques that have been used for several years in HPC systems needed to go mainstream in AI systems. Supermicro designed and manufactured the whole DLC system, including the cold plates, the coolant distribution units (CDUs) in the racks, and the chillers that provide cool water back to the equipment in the racks.<\/p>\n Notably, half of the \u201cColossus\u201d system<\/a> at xAI \u2013 comprising a total of 50,000 H100 GPUs \u2013 in its datacenter in Memphis was built by Supermicro using its DLC-1 technology. The other half of the system (with another 50,000 H100s) was built by Dell and is only air-cooled.<\/p>\n Those nodes in the Colossus machines have a pair of CPUs as well as eight of the H100 GPUs. The Server<\/a> nodes also have eight ConnectX-7 network interface cards (one for each GPU) as well as a pair of lower-speed Ethernet interface cards for system management, PCI-Express switches for linking the GPU complex to the CPUs and the on-node storage and a number of other components. The DLC-1 system used water that was 30 degrees (Celsius), and could remove about somewhere north of 70 percent of the heat out of the system, which was a big improvement in efficiency and power savings. The CDUs in the DLC-1 setup were rated at 100 kilowatts.<\/p>\n But given the dearth of power out there around the globe, and its expense, Supermicro pushed harder with the DLC-2 liquid cooling system announced this week and debuting with the Blackwell B200 GPU nodes.<\/p>\n Here is what one of these new 4U nodes with the DLC-2 cooling looks like:<\/p>\n![\"\"](\"https:\/\/www.nextplatform.com\/wp-content\/uploads\/2025\/05\/supermicro-xai-colossus-racks.jpg\")
<\/a><\/p>\n
![\"\"](\"https:\/\/www.nextplatform.com\/wp-content\/uploads\/2025\/05\/supermicro-dlc-2-sys-422GS-server.jpg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.nextplatform.com\/wp-content\/uploads\/2025\/05\/supermicro-dlc-2-racks.jpg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.nextplatform.com\/wp-content\/uploads\/2023\/05\/HPE_button_19959_V2.png\")
<\/a><\/figure>\n<\/div>\n