PCIe: Making Science Fiction Capabilities a Reality

In part 1 of this series, we discussed three technologies, CPUs, NAND based storage and PCI Express, that are making what were once considered science fiction applications, such as AI and autonomous vehicles, a reality. For many of these applications, and others, to deliver on their potential, removing data flow bottlenecks is critical.

Recognizing that the PCI Express bus had become an inhibitor to the processor’s ability to access data quickly, the PCI-SIG has worked at an aggressive pace to advance the PCI Express specification with PCIe 4.0 and PCIe 5.0 that both offer significant performance gains over PCIe 3.0. For developers, moving to the next generation of standards offers new opportunities and new challenges. To better understand some of the challenges, we’ll compare PCIe Gen 4 and PCIe Gen 5 in this post starting with PCIe 4.0.

Officially announced in June 2017, PCIe 4.0 doubled the raw bit rate of the 3.0 specification to 16 GT/s providing a total bandwidth of 64 GB/s (x16 duplex). It should be noted the Base specification was completed, with the CEM specification still in development at the time of this post.

While backward compatible to PCIe 3.0, PCIe 4.0 introduced a number of incremental changes and modified existing features that had a significant impact for designers. The following summarizes the additions and changes for PCIe 4.0.

• Speed change to 16 GT/s
• Equalization updates for 4.0 (8 GT/s to 16 GT/s)
• TSx OS changes
• 16 GT/s EIEOS
• SKP OS changes (CTRL SKP)
• Polling
• Compliance update
• 10-Bit Tag
• Data Link Feature Exchange
• Flow Control Scaling
• Rx Margining
• Retimer addition
• Configuration space register updates

With PCIe 4.0, designers encountered a few challenges in making the move from PCIe 3.0. These included insertion loss, signal integrity and comprehending tighter margin requirements which significantly affected design and validation efforts.

In comparison to PCIe 4.0, the PCIe 5.0 specification primarily focuses on the speed upgrade and the modifications in the associated physical (PHY layer). And, it should be noted, the PCIe 5.0 specification is moving very quickly. This is prompting many companies to accelerate their PCIe 5.0 development plans.

While the PCIe 5.0 specification is still in development and we’re not able to provide extensive information, there are some items that will bring new challenges for both system and PHY designers. To assist in your planning, we’ve attempted to identify areas designers should be aware of as they move forward with development efforts. The list below includes our expectations of changes from PCIe 4.0 and items we expect to remain the same.

Changes with PCIe 5.0:

• 32 GT/s data rates will result in an increase in channel loss. We estimate the system requirement will be approximately 36dB channel loss at Nyquist, with no forward error correction
• For the root complex, we anticipate two re-timers will be required in a 32 GT/s link. Without a re-timer, the root complex should work at approximately 36dB channel loss level.
• TXEQ will need to meet 500 nanosecond (ns) requirement for in-band handshaking
• A new behavioral equalizer will be implemented for 32 GT/s, featuring a 3-tap DFE and 2nd-order CTLE
• The expected eye height at end of channels with EQ will narrow increasing signal analysis complexity. Historically, the eye width has been 0.3UI. With PCIe 5.0, we expect this to remain constant.

For PCIe 5.0, a scope bandwidth (BW) of 33 GHz or higher should support design development

Consistent with PCIe 4.0:

• 128/130b encoding
• Equalization flows
• Compliance patterns
• Raw BER remains at 1E-12
• TXEQ presets
• Backward compatibility with slower speeds still expected

So, what does this mean for designers?

Where PCIe 4.0 demanded higher performance and more attention to implementation details, the PCIe 5.0 channel requirements will require new PHY equalization circuits in the transmitter (TX) and receiver (RX) improvements and sub-circuit redesigns. The increase in channel loss at 32 GT/s and, tighter jitter parameters and jitter limits, as well as return loss specifications in both the TX and RX will necessitate these actions.

With the PCIe 5.0 specification moving quickly through the approval process, companies should start their product planning process early to determine when they want to embrace the PCIe 5.0 specification. For companies working on PCIe 5.0, Tektronix’ PCIe test solutions have been designed to support the upcoming PCIe 5.0 specification. For companies planning to purchase PCIe Gen 3 or PCIe Gen 4 test equipment, Tektronix current PCIe solutions are upgradeable to support PCIe 5.0 developments. This allows you to future-proof your test and measurement investments by meeting both your current and future PCIe needs.

Addressing the challenges of testing PCIe devices requires test instruments that have protocol-aware capabilities to assist with debugging problems related to putting devices into loopback, performing link training, and understanding root cause of persistent bit error rate (BER) problems. High speed test labs require flexible test equipment to characterize and validate silicon, systems, and devices. 

To overcome these challenges, Tektronix offers solutions supporting PCIe developments from PCIe 1.0 to the upcoming PCIe 5.0 specification.  For more information, visit https://www.tek.com/wired-communications/accelerate-pcie-sas-sata-test-and-debug.

In the next part of this series, we’ll look at the new challenges of testing PCIe 5.0 designs in more detail. For additional background on PCIe 4.0, I suggest you take a look at  10 Things to Know about PCI Express.

For Part 1 of this series, visit:
https://www.tek.com/blog/pci-express-it-science-fiction-or-just-great-engineering

For Part 3, visit:
https://www.tek.com/blog/pcie-gen5-making-science-fiction-capabilities-reality