PAM4: a fundamental shift for 400G networking

PAM4 modulation is empowering and driving the transition from 100G to 400G data rates in the data centre. Its capabilities will impact the network by reducing overall 400G upgrade costs, so it is absolutely imperative that any data centre manager or director fully understands its implications when planning for future upgrades.

PAM4 concepts for consideration

Modulation

With earlier generations of 100Gbps (Gigabit per second) or lower data rates, data centre operators did not need to concern themselves as much with modulation. Transceivers simply utilized NRZ (binary non-return to zero) formatting which ensured the interoperability between data rates and interfaces.

From a technical standpoint, NRZ transmits 1 bit per waveform (0 or 1). PAM4 waveforms, by contrast, each can carry 2 bits and offer four different levels or steps, carrying 2 bits each - 00, 01, 10 or 11.

Baudrate

The use of the term baudrate may be a flashback to the dial-up modem era for some, yet it returns to our lexicon with the introduction of PAM4. Baudrate and bitrate were equivalent with NRZ modulated transmissions, but this is not the case with PAM4.

NRZ transmissions have the same baudrate and bitrate, as one symbol can carry one bit. For example, 25Gbps bitrate is equivalent to 25GBdps (gigabaud per second) baudrate.

In comparison to this, 400G PAM4 transceivers each hand off eight 50Gbps lanes on the electrical interface to the host network element. Each of these PAM4 lanes carry a line transmission rate of 25GBdpd (25-gigabaud) at 2 bits per symbol to achieve 50Gbps per lane.

Lanes and Gear Boxes

Standard 100G QSFP28 transceivers do not transmit a single lane of 100Gbps, but instead achieve 100Gbps via four lanes of 25Gbps. Housed within the transceiver are four transmitting lasers and four receivers for optical connectivity alongside four lanes of 25Gpbs on the electrical interface.

The QSFP-DD 400G Multi-Source Agreement (MSA) designates eight lanes of 50Gbps PAM4 on the electrical side (400GAUI-8) while on the optical side, they may transmit with eight lasers of 50Gbps PAM4 or four lasers of 100Gbps PAM4. An electrical gearbox converts and re-times data between the eight electrical lanes, preparing it for transmission across four lanes of 100Gbps for 4x100G transceivers. These eight lane optical transceivers do not require a gear box for re-timing.

Like baudrate, gearboxes are not new concepts and are being reintroduced for 400G applications. In the most recent designs, the gearbox is embedded into a DSP IC (Digital Signal Processor Integrated Circuit) offering additional functionalities such as equalisation. 400G QSFP-DD LR4, FR4, and LR4 transceivers each employ a gearbox to convert and re-time between electrical and optical interfaces.

Key considerations for 400G upgrades

400G port breakout connections are used at the core to aggregate connections between multiple network elements, such as servers, storage, and other appliances, with a single top-of-rack or leaf switch port. 40G and 100G breakout connections traditionally used Direct Attached Cables (DAC), Active Optical Cables (AOC), or a parallel transceiver with a Multi-fibre Push On (MPO) interface connected to four small form-factor (SFP) type transceivers, as modulation was not a concern in these previous cycles.

Reduce upgrade expenses by deploying alongside existing 100G network elements

Legacy 100G network elements accept four lanes of 25G NRZ communication at the electrical interface. Standard 400G 4x100G breakout DAC cables split each breakout section into a 2x50G PAM4 electrical signal but do not offer a gearbox function to re-time to the 4x25G NRZ interface accepted by legacy switches. 4x100G AOCs are unfortunately expected to follow a similar path in not offering backward interoperability with legacy 100G NRZ switches.

Legacy transceivers are built on four lanes of 25G NRZ formatted signalling on both the optical and electrical side. Common NRZ four lane transceivers SR4, LR4, etc. will not be interoperable with 400G transceivers in a 4x100 application.

Luckily, some transceiver-based breakout solutions offer options for interoperability with legacy gear. In particular, a new class of single lambda (single wavelength) 100G transceivers has been introduced that will support 100G PAM4 at the optical interface and 4x25G NRZ at the electrical interface. These transceivers perform the re-timing between PAM4 and NRZ modulation schemes within the transceiver gearbox.

The QSFP28 DR and FR (also referred to as DR1 and FR1) transceivers are the first transceivers of this kind and are already available. These optics are fully interoperable with not only legacy 100G network gear but also with QSFP-DD DR4 and DR4+ breakout transceivers. The QSFP-DD DR4 and DR4+ are parallel series modules, accepting an MPO-12 connector, with breakouts to LC connectors to interface with the DR or FR transceivers. DR4 to 4xDR connections are up to 500M, DR4+ to 4xFR connections are up to 2km over single-mode fibre.

The network is not ready for PAM4 but there is still a strong need to increase port density

The QSFP-DD specification used by 400G network elements does offer backward compatibility to NRZ technology. Network elements using these additional capabilities of the QSFP-DD MSA offer flexibility for data centre operators to increase port density in comparison to legacy 100G QSFP28. 200G, 2x100G DAC and transceiver solutions are available for these use cases.

2x100G DAC cables offer a 200G, 8x25G NRZ electrical interface to a network element’s QSFP-DD port. The corresponding breakout cable’s terminations connect with 100G NRZ port in legacy equipment. These DAC cable applications are only practical for in-rack connections.

Copper DAC cables are less complex in design and construction, making them the first products to market for this application, however in line with ever-changing demands breakout AOC cable technologies continue to be developed and are anticipated to be available on the market soon.

200G 2x100G transceivers connect with legacy NRZ transceivers at reaches of up to 10km. These transceivers act as “two transceivers in one housing” complete with two sets of Tx/Rx pairs on the faceplate of each unit.

Multiple 2x100G optical choices are available for these requirements, depending on the reach the application calls for.

The short-reach option is to opt for the QSFP28-DD 2x100G SR4 which utilises a MPO-24 connector. The SR4 is capable of supporting reaches of up to 100m multimode fibre (MMF) while doubling port density over a standard QSFP28 SR4 pluggable optic.

For those needing a bit more oomph, the intermediate and long-reach alternatives are the QSFP28-DD 2x100G CWDM4, for a reach of up to 2km or theQSFP28-DD-2x100G LR4 for infrastructure reach requirements of up to 10km single-mode fibre (SMF). The LR4 transceivers leverage the use of a new high-density “CS” connector which encompasses a full Tx/Rx pair within the footprint of a single LC connector. The CS connector enables the two Tx/Rx pairs from a single transceiver.

Understanding and choosing your upgrade path

Without a doubt, multimode fibre has the largest footprint in today’s data centres. However, single mode fibre is rapidly growing as a viable option for many data centre connections. The technical requirements for 100G and 400G higher data rates are pushing the capabilities of vertical-cavity surface-emitting lasers (VCSEL) that may limit the reach in comparison to previous technologies.

These practical limitations in the data centre will be raising their head when upgrading over a multimode fibre pair. 40G and 100G transceiver technologies deploy both standard-based and proprietary technologies to transmit or receive on a fibre pair.

With a multitude of upgrade options available to network managers and data centre operators, it is of the utmost importance to plan as far in advance as possible. Truly understanding each upgrade path available and choosing the right one for your business can minimise or even avoid complications down the line. Switching plans mid-deployment can have major time and cost implications and should be avoided where foreseeable at all costs.

Figure 1 - Roadmap of the upgrade paths between your current network and one that serves the needs of tomorrow's consumers

Preparing for the advancements of future data centres

While PAM4’s arrival to the market brings forth promising transmission methodologies that achieve the speeds and reaches that 4K video, 5G mobile, and IoT all demand, it also brings forth a host of complexities that must be carefully navigated. Taking the time to thoughtfully map out which upgrade path, and which optics within those paths, makes the most sense for your business can avoid wasted time and capital.

Whether your needs are to reduce operating expenses, increase port density, get prepared for tomorrow’s bandwidth or speed needs, or even something in between, PAM4 and QSFP-DD solutions can help you win tomorrow’s upgrade battles, today.