Uncovering the hidden costs of cooling

Cooling technology is advancing at a pace with the emergence of new technologies such as hybrid cooling, which has the potential to significantly reduce energy consumption and the impact of data centres on the environment. With cooling accounting for up to 40% of the energy used in data centres, and one of the biggest outlays for operators, there is a need to understand the hidden costs associated with the various technologies and how they compare.

DX cooling

During the past 15-20 years, DX has become the dominant cooling solution and, until recently, this was driven by the fact that it was considered cheap and simple to install. DX units offer a number of benefits, such as good levels of cooling, coupled with a low footprint. As these systems are based on indirect cooling, there is also no danger of introducing contaminants from outside into the data centre. However, DX systems are less desirable from an energy efficiency perspective as well as their dramatic instability, in terms of refrigerant pricing, meaning that these units are no longer the low-cost option that they once were. 

Chilled water cooling

Chilled water systems have also come to the fore in recent years, as the capacities in data centres have steadily increased. However, hybrid solutions offer the advantage of having a lower refrigerant charge per unit – despite the fact that these operate using a similar process to a chiller. Stulz UK’s largest hybrid solution, for example, uses 7.6kg of refrigerant per circuit (with a twin circuit). An equivalent, conventional chiller system will use around 40% more refrigerant on a 1:1 basis. On large systems it is also important to note that, if a chiller fails, there is a greater risk of losing a significant amount of refrigerant. With a hybrid unit, the capacity is spread over multiple units, but the risk of losing large quantities of refrigerant is significantly reduced. 

Hybrid cooling

Hybrid cooling combines the reliability and control of a DX system, with the energy saving benefits of a free cooling system. In warmer months, when the external ambient temperature is above 20°C, the Stulz GE system operates as a water-cooled DX system and the refrigeration compressor rejects heat into the water circuit via a plate heat exchange (PHX) condenser. The water is pumped to an air blast cooler where it is cooled, and the heat rejected into air.

In cooler months, below 20°C external ambient temperature, the system automatically switches to partial free cooling mode known as “mix mode”. In this mode, the water is directed through both proportionally controlled valves and enables proportional free cooling and water-cooled DX cooling to work together, with the dry cooler fans being used to cool the water to the desired level to achieve the required cooling capacity. In the winter months, dependant on water temperature and/or heat load demands, the water can be used in ‘free cooling mode’. In this mode, the water is directed through the free cooling coil only, eliminating the need for mechanical cooling (typically 3 degrees below setpoint). As the technology also uses indirect cooling, there is no risk of introducing contaminants from outside into the data centre. 

Mix mode cooling

Mix mode cooling is the point between free cooling and DX. This means the ambient air outside is cold enough to pre-cool the water provided to the hybrid unit but not quite cold enough to lower the temperature fully, to the required parameters. (Typically this is 3-6 degrees below the internal setpoint – dependant on system efficiency.) This ‘sweet spot’ requires an element of mechanical cooling as a ‘top up’ to meet the load within a room. This mix mode cooling typically makes up between 50-68% of the potential operation mode of a system.

Data centre operators can extend the free cooling and mix mode potential of their hybrid equipment by raising the return air temperatures. Potentially, it is possible to increase the return air from 24 degrees to 27 degrees, resulting in additional cost savings of up to 40% per year.

This hybrid approach drastically reduces power consumption over traditional DX systems and hence data centres can achieve huge cost savings. Data centre operators are also starting to scrutinise the pay back on their investments and this is where hybrid approaches can offer significant benefits. Payback depends on the size of the system and location, but the return on investment can be very quick at around 3-4 years for a medium sized data centre (circa 500kW). The larger hybrid systems offer the quickest payback periods, however.

The hidden cost of refrigerant

The initial CAPEX cost for a hybrid cooling system is higher than a traditional computer room air conditioning (CRAC) unit. But there are other factors that need to be considered, when calculating the total cost of ownership and OPEX. As part of the EU Energy Strategy, there is a need to reduce fluorinated greenhouse gas (F gas) by -60% by 2030. The latest regulation, which came into force in 2015, means that refrigerant availability and cost are being heavily affected with a revised target due in 2020 and yet further rises in refrigerant costs expected. As refrigerant is now a very expensive ingredient, this can have a significant impact on the total cost of ownership of the cooling technology.

For example, in August 2017 the average purchase price of R407c and R410a refrigerants was £14.78 and £29.78 respectively per kilo. By the end of July 2018, the average purchase price of R407c and R410a both increased to £50.66 and £58.59 respectively per kilo, and further increases followed shortly after. If we look at a 50m (equivalent length) and a total heat rejection capacity of 80kW, the cost of refrigerant equates to a cost of just over £3,600 per system, rising from £1,800 – representing an increase of 98% in just one year. 

It is not just the hidden cost of refrigerant that poses a potential burden to data centres, however. In the future, there will be an intensification of certification for operating companies and staff, as well as a need for record keeping. Records will need to be kept for a minimum of five years and must include types of F-Gas used, quantity and how it is recycled when removed. It is therefore important to think about using sustainable solutions, today, and to reduce the use of refrigerants where possible.

Hybrid solutions require significantly less quantities of refrigerant and against this back-drop of rocketing refrigerant prices, this will be a significant advantage.

Refrigerant leaks: the legal requirements

It is important to note that data centre operators will be held responsible for preventing refrigerant leaks from their equipment, in the future. Contractors that install, maintain or dispose of equipment will share this responsibility. However, for equipment that contains F gas, above certain thresholds, there will be a requirement to check cooling equipment for leaks at specific intervals (further information is available at: https://www.gov.uk/guidance/f-gas-in-refrigeration-air-conditioning-and-fire-protection-systems ). In short, this means that maintenance can no longer be a reactive response, but a legal requirement. These checks must form part of a preventive maintenance strategy.

New low GWP refrigerants

There has been significant interest in the potential for a new variety of refrigerants with low global warming potential (GWP). The new ‘super refrigerants’ currently being reviewed by the data centre cooling industry are R1234ze and R1234yf. Unfortunately, the development of a low GWP refrigerant CRAC DX solution has been slow to come to market and there are trade-offs that will need to be considered in the future.

It is likely that the initial cost of low GWP refrigerants will be higher and flammability will also require careful risk assessment – butane, for example, is a good refrigerant, but is incredibly dangerous. New refrigerants will face similar issues and will come under increasing scrutiny. A large, single chiller using 200kg of refrigerant presents a much greater risk of leaks, than when using a hybrid system, comprising multiple small units – each using 18kg of refrigerant. Data centre operators are going to become increasingly aware of this issue as regulations around refrigerants continue to be strengthened.

Conclusion

Next Generation Data (NGD) and other high-profile data centres are now adopting hybrid technology, and data centre managers are finding that they can save costs, maintain their green credentials and ensure a high level of resilience. As the cost of refrigerant continues to rise, conventional technologies will become increasingly expensive to operate. In the future, hybrid cooling could help reduce the risks associated with refrigerants, while taking advantage of sustainable free cooling approaches.