Cyclone V and some transceiver CDR/PLL parameters
Introduction
Connecting an Intel FPGA (Altera) Cyclone V’s Native Transceiver IP to a USB 3.0 channel (which involves a -5000 ppm Spread Spectrum modulation), I got a significant bit error rate and what appeared to be occasional losses of lock. Suspecting that the CDR didn’t catch up with the frequency modulation, I wanted to try out a larger PLL bandwidth = track more aggressively at the expense of higher jitter. That turned out to be not so trivial.
This post sums up my findings related to Quartus. As for solving the original problem (bit errors and that), changing the bandwidth made no difference.
Toolset: Quartus Lite 15.1 on Linux.
And by the way, the problem turned out to be unrelated to the PLL, but the lack of an equalizer on Cyclone V’s receiver. Hence no canceling of the low-pass filtering effect of the USB 3.0 cable. I worked this around by setting XCVR_RX_LINEAR_EQUALIZER_CONTROL to 2 in the QSF file and the errors were gone. However this just activates a constant compensation high-pass filter on the receiver’s input (see the Cyclone V Device Datasheet, CV-51002, 2018.05.07, Figure 4) and consequently works the problem around for a specific cable, not more.
Assignments in the QSF file
In order to change the CDR’s bandwidth, assignments in the QSF are due, as detailed in V-Series Transceiver PHY IP Core User Guide (UG-01080, 2017.07.06) in the section “Analog Settings for Cyclone V Devices” and on page 20-28. In principle, CDR_BANDWIDTH_PRESET should be set to High instead of its default “Auto”. In this post, I’ll also set PLL_BANDWIDTH_PRESET to High, even though I’m quite confident it has nothing to do with locking to data (rather, it controls locking to the reference clock). But it causes quite some confusion, as shown below.
So all that is left is to nail down the CDR’s instance name, and assign it these parameters.
Now first, what not to do: Using wildcards. This is quite tempting because the path to the CDR is very long. So at first, I went for this, which is wrong:
set_instance_assignment -name CDR_BANDWIDTH_PRESET High -to *|xcvr_inst|*rx_pma.rx_cdr
set_instance_assignment -name PLL_BANDWIDTH_PRESET High -to *|xcvr_inst|*rx_pma.rx_cdr
And nothing happened, except a small notice in some very important place of the fitter report:
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ; Ignored Assignments ; +--------------------------------------------------+---------------------------+--------------+------------------------------------------------------------+---------------+----------------------------+ ; Name ; Ignored Entity ; Ignored From ; Ignored To ; Ignored Value ; Ignored Source ; +--------------------------------------------------+---------------------------+--------------+------------------------------------------------------------+---------------+----------------------------+ ; Merge TX PLL driven by registers with same clear ; altera_xcvr_reset_control ; ; alt_xcvr_reset_counter:g_pll.counter_pll_powerdown|r_reset ; ON ; Compiler or HDL Assignment ; ; CDR Bandwidth Preset ; myproj ; ; *|xcvr_inst|*rx_pma.rx_cdr ; HIGH ; QSF Assignment ; ; PLL Bandwidth Preset ; myproj ; ; *|xcvr_inst|*rx_pma.rx_cdr ; HIGH ; QSF Assignment ; +--------------------------------------------------+---------------------------+--------------+------------------------------------------------------------+---------------+----------------------------+
Ayeee. So it seems like there’s no choice but to spell out the entire path. I haven’t investigated this thoroughly, though. Maybe there is some form of wildcards that would work. I also discuss this topic briefly in another post of mine.
So this is more like it:
set_instance_assignment -name CDR_BANDWIDTH_PRESET High -to frontend_ins|xcvr_inst|xcvr_inst|gen_native_inst.av_xcvr_native_insts[0].gen_bonded_group_native.av_xcvr_native_inst|inst_av_pma|av_rx_pma|rx_pmas[0].rx_pma.rx_cdr set_instance_assignment -name PLL_BANDWIDTH_PRESET High -to frontend_ins|xcvr_inst|xcvr_inst|gen_native_inst.av_xcvr_native_insts[0].gen_bonded_group_native.av_xcvr_native_inst|inst_av_pma|av_rx_pma|rx_pmas[0].rx_pma.rx_cdr
I guess this clarifies why wildcards are tempting.
Verifying something happened
This is where things get confusing. Looking at the fitter report, in the part on transceivers, this was the output before adding the QSF assignments above (pardon the wide line, this is what the Fitter produced):
; -- Name ; frontend:frontend_ins|xcvr:xcvr_inst|altera_xcvr_native_av:xcvr_inst|av_xcvr_native:gen_native_inst.av_xcvr_native_insts[0].gen_bonded_group_native.av_xcvr_native_inst|av_pma:inst_av_pma|av_rx_pma:av_rx_pma|rx_pmas[0].rx_pma.rx_cdr ; ; -- PLL Location ; CHANNELPLL_X0_Y49_N32 ; ; -- PLL Type ; CDR PLL ; ; -- PLL Bandwidth Type ; Auto (Medium) ; ; -- PLL Bandwidth Range ; 2 to 4 MHz
And after adding the QSF assignments:
; -- Name ; frontend:frontend_ins|xcvr:xcvr_inst|altera_xcvr_native_av:xcvr_inst|av_xcvr_native:gen_native_inst.av_xcvr_native_insts[0].gen_bonded_group_native.av_xcvr_native_inst|av_pma:inst_av_pma|av_rx_pma:av_rx_pma|rx_pmas[0].rx_pma.rx_cdr ; ; -- PLL Location ; CHANNELPLL_X0_Y49_N32 ; ; -- PLL Type ; CDR PLL ; ; -- PLL Bandwidth Type ; High ; ; -- PLL Bandwidth Range ; 4 to 8 MHz
Bingo, huh? Well, not really. Which of these two assignments made this happen? CDR_BANDWIDTH_PRESET or PLL_BANDWIDTH_PRESET? In other words: Does the fitter report tell us about the bandwidth of the PLL on the reference clock or the data?
The answer is PLL_BANDWIDTH_PRESET. Setting CDR_BANDWIDTH_PRESET doesn’t change anything in the Fitter report at all. I know it all too well (after spending some pleasant quality time trying to figure out why, before realizing it’s about PLL_BANDWIDTH_PRESET).
So where’s does CDR_BANDWIDTH_PRESET do its trick?
To find that, one needs to get down to the post-fitting properties of the rx_cdr instance. The following sequence applies to Quartus 15.1′s GUI:
After fitting, select Tools > Netlist Viewers > Technology Map Viewer (Post-Fitting). Locate the instance in the Find tab (to the left; it’s a plain substring search on the instance name given in the QSF assignment). Once found, click on the block in the graphics display so its bounding box becomes red, and then right-click this block. On the menu that shows up, select Locate in Resource Property Editor.
And that displays a list of properties (which can be exported into a CSV file). One of which is rxpll_pd_bw_ctrl. Changing CDR_BANDWIDTH_PRESET to High altered this property’s value from 300 to 600. Changing it to Low sets it to 240.
And by the way, a change in PLL_BANDWIDTH_PRESET to High has no impact on any of the properties listed in the Resource Property Editor for the said instance, but making it Low takes pfd_charge_pump_current_ctrl from 30 to 20, and rxpll_pfd_bw_ctrl from 4800 to 3200. Whatever that means.
It’s worth mentioning that the CDR is instantiated as an arriav_channel_pll primitive (yes, an Arria V primitive on a Cyclone V FPGA) in the av_rx_pma.sv module (generated automatically for the Transceiver Native PHY IP). One of the instantiation parameters is rxpll_pd_bw_ctrl, which is assigned 300 by default. The source file doesn’t change as a result of the said change in the QSF file. So the tools somehow change something post-synthesis. I guess.
There are however no instantiation parameters for neither pfd_charge_pump_current_ctrland nor rxpll_pfd_bw_ctrl. So the rxpll_pd_bw_ctrl naming match is probably more of a coincidence. Once again, I guess.
A closer look on the PLL
It’s quite clear from above that CDR_BANDWIDTH_PRESET influenced rxpll_pd_bw_ctrl (note the _pd_ part) and that PLL_BANDWIDTH_PRESET is related to a couple of parameters with pfd them. This terminology goes along with the one used in the documentation (see e.g. Figure 1-17, “Channel PLL Block Diagram” in Cyclone V Device Handbook Volume 2: Transceivers, cv_5v3.pdf, 2016.01.28): The displayed terminology is that PFD relates to the Lock-To-Reference loop, which locks on the reference clock, and PD relates to the Lock-To-Data loop, which is the CDR.
This isn’t just a curiosity, because the VCO’s output dividers, L, are assigned separately for the PD and PDF loops (see the fitter report as well as Table 1-9).
As for the numbers in the fitter report, they match the doc’s as shown in the two relevant segments below. The first relates to a Native PHY IP, and the second to a PCIe PHY, both on the same design, both targeted at 5 Gb/s (and hence having the same “Output Clock Frequency”).
; -- Reference Clock Frequency ; 100.0 MHz ; ; -- Output Clock Frequency ; 2500.0 MHz ; ; -- L Counter PD Clock Disable ; Off ; ; -- M Counter ; 25 ; ; -- PCIE Frequency Control ; pcie_100mhz ; ; -- PD L Counter ; 2 ; ; -- PFD L Counter ; 2 ; ; -- Powerdown ; Off ; ; -- Reference Clock Divider ; 1 ;
versus
; -- Reference Clock Frequency ; 100.0 MHz ; ; -- Output Clock Frequency ; 2500.0 MHz ; ; -- L Counter PD Clock Disable ; Off ; ; -- M Counter ; 25 ; ; -- PCIE Frequency Control ; pcie_100mhz ; ; -- PD L Counter ; 1 ; ; -- PFD L Counter ; 2 ; ; -- Powerdown ; Off ; ; -- Reference Clock Divider ; 2 ;
In both transceivers, a 2500 MHz clock is generated from a 100 MHz reference clock. It seems like the trick to understanding what’s going on is noting footnote (2) of Table 1-17, saying that the output of L_PD is the one that applies when the PLL is configured as a CDR.
In the first case, the reference clock is fed into the phase detector without division. Since the reference clock is not divided, 100 MHz reaches one input of the phase detector. As the output is divided by PFD_L = 2 and then by M=25, the VCO has to run at 5000 MHz so that its output divided by 50 matches the 100 MHz reference. That doesn’t seem very clever to me (why not pick L=1, and avoid 5 GHz, which I’m not even sure is possible on that silicon?). But at least the math adds up: The output is divided with PD_L = 2, and we have 2500 MHz.
Now to the second case (PCIe): The reference clock is divided by 2, so the phase detector is fed with a 50 MHz reference. The VCO’s clock is divided by PDF_L = 2 and then with M = 25, and hence the VCO runs at 2500 MHz. This way, the total division by 50 (again) matches the 50 MHz reference on the phase detector. PD_L = 1, so the VCO’s output is used undivided, hence an output clock of 2500 MHz, again.
I’m not sure that I’m buying this explanation myself, actually, but it’s the only way I found to make sense of these figures. At some point I tried to convince the tools to divide the reference clock by 2 on the Native PHY (first case above) by adding
set_instance_assignment -name PLL_PFD_CLOCK_FREQUENCY "50 MHz" -to "frontend:frontend_ins|xcvr:xcvr_inst|altera_xcvr_native_av:xcvr_inst|av_xcvr_native:gen_native_inst.av_xcvr_native_insts[0].gen_bonded_group_native.av_xcvr_native_inst|av_pma:inst_av_pma|av_rx_pma:av_rx_pma|rx_pmas[0].rx_pma.rx_cdr"
to the QSF file. This assignment was silently ignored. It wasn’t mentioned anywhere in the reports (not even in the Ignored Assignments) part, but the Divider remained at 1. I should mention that this assignment isn’t documented for Cyclone V, but Quartus Assignment Editor nevertheless agreed to generate it. And Quartus usually refuses to load a project if anything is fishy in the QSF file.
