The RIKEN research institute's supercomputer Fugaku, hailed as the world's fastest, produces a huge amount of heat as its central processing units (CPUs) blaze through calculations. While the supercomputer is expected to play a valuable role in cutting-edge research, a peek behind its dazzling performance reveals a battle to beat the heat.
The building housing Fugaku is in Kobe's Chuo Ward, and steam produced during the cooling process billows from a cooling tower on the roof -- almost like the scene at a power plant. In fact, according to Riken, there is so much steam that even since last spring there have been several cases in which people staying at a nearby hotel mistook it for smoke and alerted police and firefighters.
Toshiyuki Tsukamoto, a deputy division director at the RIKEN Center of Computational Science, explained, "The heat generation density is on par with that of a nuclear reactor. Depending on the type of calculation, power consumption varies considerably, and the heat value changes in tandem with this. It is only supercomputers that require measures against such massive changes in heat generation."
Fugaku, built to succeed the "K" supercomputer taken offline in August 2019, is due to commence full-scale operations in March this year. In tests last year, it was used to simulate the dispersal of droplets in formulating measures against the coronavirus. It has claimed the top spot in four categories in a biannual world ranking of supercomputer performance two times in a row.
In the computing room on the third floor of the building housing Fugaku, 432 racks hold over 160,000 central processing units, which generate heat when they operate. The heat density can rise to over 100 kilowatts per square meter. This is like having 100 household electric heaters operating within a 1-meter-square space.
To make sure the CPUs can run efficiently, they must be kept under 30 degrees Celsius, but without cooling, their temperature would rise above 100 C in a matter of seconds. To prevent this, the supercomputer is equipped with a large water-based liquid cooling unit. The cooling system has primary and secondary branch pipes through which water flows to remove heat from the CPUs.
It is the secondary water system that directly cools the approximately 160,000 CPUs. It carries water at about 15 C close to the CPUs to remove their heat, which in turn increases the temperature of the water to between 19 and 25 C. This water is then sent to heat exchangers. There, the water in the secondary system is cooled back down to about 15 C by the water in the primary system, which is equipped with up to 11 cooling devices. Once cooled, the secondary system water is pumped back to the CPUs.
The reason the primary and secondary water systems are separated is to prevent impurities from building up and blocking the thin secondary system pipes. The water in the secondary system is purified and contains an anti-corrosive substance, while industrial water is pumped around the primary system.
The most difficult task, Tsukamoto reflects, was dealing with temperature fluctuations. Different types of calculations result in detailed changes in the CPUs' operation, resulting in considerable changes in the amount of electricity consumed and the amount of heat produced. This can produce surges of heat in as little as 1/1000th of a second.
To address this, Tsukamoto and other researchers simulated the flow of water in complicated pipe layouts to devise a system that could cope even if the unexpected occurred. They installed sensors to monitor the temperature of the water in the secondary system and enabled the flow of water in the primary system to be adjusted to constantly keep the temperature of the secondary system water to about 15 C.
"When there are sudden rises and falls in temperature in a short time, the question of how to continually cool the water becomes important. We had no past cases to refer to, and we thought about how to put together existing technology," Tsukamoto said.
Tsukamoto has been involved in the operation of Fugaku after having worked with K. "It's only natural that it's working properly," he said. "I do what's natural as if it is only natural. That's a source of pride for those working behind the scenes, and I think it's also something I'm confident about."
It is apparent that such efforts to advance sophisticated cooling technology have helped propel Fugaku to the forefront of supercomputing.
(Japanese original by Mirai Nagira, Science and Environment News Department)