What’s the outlook for HDDs?

According to IDC forecasts¹, over 22ZB of storage capacity will be delivered to the market between 2018 and 2025. It is predicted that around 26% of that capacity will be fulfilled by flash technology. However, established HDD technology will contribute by far the most significant portion at 59%. With an expected increase in AI applications and support for innovative new automotive services coupled with the roll-out of 5G, it is essential that enterprise and datacenter operators review all possible storage options in order to make optimal investments for the years ahead.

Obviously, flash-based storage, such as SSDs, are very attractive for a wide range of applications thanks to their small form-factor, low power, low latency and high throughput. However, these advantages come at a significant price premium, typically still 8-10 times higher in cost compared to equivalent HDD storage. As the boot drive for a server or a tier-0 application, the additional cost can be justified and this is not an application space where HDDs are likely to be considered suitable again. But in terms of planning, it is essential to consider whether SSDs are likely to drop far enough in price to make them attractive as an alternative in applications where HDDs are in use today.

Looking historically, the price/TB of both SSDs and HDDs has fallen over the past years. However, plotting these prices side-by-side indicates that SSD technology price reductions are dropping at the same rate as that of HDDs. This means that, for the foreseeable future, SSDs are unlikely to attain a price point that makes any increase in their unit capacity look more attractive than the HDD alternative whose price will also have improved in the same timeframe.

The move to Industry 4.0, growth in Internet of Things applications, and the subsequent handling of machine data will mean that the quantity of data per second being moved to cloud services will be around 160 times more per second than that generated today by largely human-operated apps and cloud services. It would be assumed that the speed of SSDs would disqualify an HDD-based solution in such cases, but this is also not always the case.

Thanks to research undertaken by Toshiba, solutions with many low-cost HDDs in a RAID10 configuration or a parallel software defined architecture have been shown to outperform a cost-comparable solution with a few SSDs in terms of IOPS (for block sizes larger than 64k Byte) while offering three times more storage capacity. This demonstrates that the many parallel spindles of HDDs in an appropriate multi-drive configuration can still compete with SSDs. Increasingly datacenters will also need to implement strategies for their data storage, preselecting what data should be stored and what shouldn’t, or even compressing that data as it arrives. Thought also needs to be applied to a deletion strategy, understanding when stored data is no longer of use and can be discarded.

It is also worth considering that, in 2018, the quantity of flash storage shipped was equivalent to around 10 – 15% of the capacity shipped as HDDs². There is currently significant investment of $100B³ or more being made in fabrication facilities that will allow flash-based memory manufacturers to double their output in the coming years but, by the time this capacity comes online, it will still only supply one quarter of today’s total delivered storage capacity. It is clear that, for now at least, flash storage manufacturing capacity will not grow fast enough to cover demand or reach the point where the price/TB will be significantly impacted.

HDDs are a mature technology by comparison and many of the improvements that can be offered in the coming years are incremental in comparison to the leaps and bounds seemingly being made in flash-memory storage. One area of impressive improvement is the capacity per drive, with today’s largest drives expected to gain an additional 2 TB of storage each year for the next few years. Toshiba’s expertise in bringing helium sealed nearline HHDs to market has seen such drives already grow from 14 TB in 2017, to 16 TB in 2019. One side effect of the use of helium is that it has enabled the platters to be made thinner, providing space for nine platters.  Future iterations of this technology are likely to see up to 10 platters integrated into a single 20+ TB drive.

A further advantage of the move to helium is the reduction in power consumption it also offers, a parameter of significant interest to datacenter teams. It is not unusual for storage servers that previously consumed kW of energy to now require only a few hundred Watts. Today’s most efficient HDDs need around 30 – 50% less energy than their equivalent forebears. Datacenter solutions also need to be able to power down drives that are not in use to further optimize power use.

Exchanging out 2 – 3 TB drives for newer devices helps to reduce energy consumption while also providing increased capacity for a similar capital investment. This is essential bearing in mind that it is estimated that cloud/online data storage consumes 500 MW more energy each year, equivalent to the output of half a nuclear power station or 80 windmills. Of course, none of these improvements can be allowed to impact on reliability and the mean-time to failure (MTTF) that need to be maintained or improved upon too.

Intelligent surveillance solutions are another significant driver of storage growth and is also another application being impacted by AI technologies. Here the requirements diverge significantly from the traditional datacenter, with drives needing to support a temperature range of 0°C to 70°C while also providing excellent MTTF figures and acceptable warranty periods. New camera systems delivering higher-resolution video streams generate even more data than ever before. But drive technology also has to ensure that this data can be written without dropping frames, while also providing access to that data for the purpose of analysis by AI and other big data applications.

Outside of the sphere of enterprise datacenters, consumers and SMEs are also feeling overwhelmed with the quantity of data they are having to deal with. This ranges from photos and videos, to business data and design files for complex projects. Toshiba’s N300 range of helium-filled HDDs brings 16 TB of storage to the masses, allowing high capacity NAS solutions to be operated at the home or office as a local backup or alternative to cloud storage solutions. It also provides an improved level of privacy for those unwilling to commit all of their personal data to the cloud.

What is certain is that demand for storage will, over the next 5 – 10 years, continue to grow rapidly. Big data projects along with the infant AI market will be a big part of this future demand as currently unforeseen and in-development, such as in the area of automotive and 5G-connected devices, develop and roll-out new applications. Despite planned advances in flash technology and available fabrication capacity, SSDs will remain a luxury item that will be used when the budget can be justified, even though the price/TB will continue to fall. This is because the price/TB of HDDs will also continue to fall at the same rate. The technology of HDDs is also continually advancing and still has advancements and improvements to share. As such, they will continue to provide the bulk of storage capacity, their power consumption improvements will ensure they remain attractive, and, in certain RAID or software defined storage configurations, they will remain competitive or better than SSDs for the same budget in terms of performance.

Reference 1: IDC DataAge 2025 whitepaper

Reference 2: Gartner Research, Market Share Analysis: Solid-State Drives, Worldwide, 2018