Background
More than anything, this is a reminder to self why I must use a temporary credit card number when I’ll renew my EV Code Signing certificate I purchased from ssl.com, a few years from writing this (if they’ll still exist by then). I need it for Microsoft’s Attestation signing of drivers for Windows, as detailed on this separate post of mine.
The business of selling certificates to websites have seen better days since Let’s Encrypt began to offer such for free, so in a way I can understand that a company like ssl.com needs to increase its revenues on the higher end of certificates. But it seems like they’ve lost control over the machine that charges money from credit cards. Or maybe they’re struggling for cash with nothing to lose, so it could be that there will be no ssl.com to renew the certificate with. Actually, do credit card companies offer protection from companies that go bankrupt?
May 2024 update: ssl.com did it again, as described in a new post.
The story until things got wrong
Back in May 2021, I purchased a 3-year EV Code Signing certificate from ssl.com, paying 747 USD with my regular credit card. The vetting process went reasonably smooth, and I got the certificate a couple of weeks after issuing the order, which makes sense given the extensive evaluation required.
Along with the congratulation mail from ssl.com, I got an offer to use cloud signing with eSigner.com for free. That’s a web utility, allowing signing a file by dragging it into the browser, and subsequently validating oneself with the mobile phone. It’s indeed more convenient that using the Yubikey USB dongle that is physically sent from ssl.com after purchasing a certificate. So far so good.
Then in July 2021, I got a mail from ssl.com thanking me for participating in public beta of eSigner (I did?) and also asking me to participate in a follow-up survey in exchange for “100 free signatures after the free beta period ends”. And so I did. It was quick and painless.
And then on August 20th 2021, another email thanking me for participating in the eSigner Cloud Signing Beta Project. It also said that “Beginning on September 1, SSL.com will offer eSigner as a paid service available to all Document Signing and EV Code Signing customers. Current Beta participants with low to medium usage will automatically be placed in the Tier 1 group with overage fees waived”.
Frankly speaking, I didn’t read that long. Knowing I had 100 free signatures, I couldn’t care less.
And just to have this clear: There wasn’t a single word in this email about being charged, let alone mentioning a sum.
Who charged me 100 USD?
They didn’t wait long. A couple of weeks after the last email, I saw a 100 USD charge on my credit card by “SSL.COM CLOUD SERVICES HTTPSWWW.SSL”. I wasn’t even sure if this had to do with ssl.com, because the latter used a slightly different name.
Did I agree to this? Of course I didn’t. I didn’t know anything about this until I saw this on my credit card bill. So apparently, they thought it was fine informing me that I was about to join “Tier 1″, and from that I should have figured what that means: A completely ridiculous program, offering 10 eSigner signatures per month, with a monthly price of 100 USD. In other words, pay 100 USD a month for sparing yourself the effort of using the USB dongle. Max 10 times a month.
This is credit card fraud by definition. The only reason I didn’t cancel it through the credit card company was that it requires canceling the credit card altogether. That would mess up recurring payments, so trying to resolve this with sll.com got priority.
No big deal, just click “revoke”
So I dropped them a mail. The response was to read this KB article on how to cancel an eSigner account. I’ll give you the highlight: A click on a link saying “REVOKE”. Given that I had already signed things with this certificate, I wasn’t at all happy clicking on that link. Even though customer support told me this doesn’t revoke the certificate in the usual sense (i.e. invalidates its use retrospectively).
In fact, I would put a beer on that the use of the R-word was made intentionally to scare.
I could also add that the use of the certificate was only for attestation signing, so it was only checked when I submitted the drivers — the drivers themselves were signed by Microsoft, with their signature.
All in all, there should in theory not be any issue even if the certificate was revoked, but I really wasn’t in the mood of pushing my luck on this one.
Besides, given that I happened to have 100 free signings, there’s no justification to ask me to cancel my eSigner account. Don’t get me wrong: It’s not like I’m going to get anywhere close eSigner in the future. I just don’t want to mess things up.
Customer non-support
Another thing the response from ssl.com said was “I have also created a refund request ticket for the billing team to refund the charge. Please stand by for update from the billing team”.
That update failed to arrive. Also, when I pointed out the 100 free signings issue, I got “The refund request is for the 100 dollar charge. If you do not use the 100 signings, you may be subject to another charge next month but the billing team would have to remove the charge. Please let me know if I can be of further assistance”.
A couple of weeks after not getting any update on the refund, I sent a nag mail, and got “This has been escalated to the billing team for investigation. Please stand by for update from the billing team”. And hey, Win from the Billing Department finally responded and told me he would issue a refund and inform me when that was done. And so he did. That is, he informed me about the refund, but I didn’t see it.
And then I was charged again on November 1st, 100 USD again. Why not?
Looking at the billing details for October, it turned out that Win had indeed made a refund: He had effectively canceled the 100 USD charged in October, but not the one in September. So nobody ever stopped the machine charging 100 USD each month. All I got was a refund for the charging in the middle.
All in all, 200 USD nicked from my credit card. And counting.
Bottom line
I mentioned the conclusion above: Always use a temporary credit card for payments abroad, and don’t even trust a company whose core business is trust. In the past, it was possible to cancel a payment made abroad without canceling the credit card, but the rules have changed.
Epilogue
After writing this post, I kept on nagging with a couple of more emails to ssl.com, and somehow the second one caught their attention. Without getting into the details of the email exchange, it was clear that the game had changed. I saw a refund for the remaining 200 USD quite immediately. Plus a credible promise that my credit card won’t be charged again.
So while I still stand behind every word I’ve written above, it more than appears that my specific case was resolved in a good way.
Introduction
This post summarizes some scattered findings I made while trying to make a Cyclone 10′s signal detect feature work properly for detecting a SuperSpeed USB LFPS signal. As it turned out, Cyclone 10′s transceiver isn’t capable of this, as explained below.
But since the documentation on this issue was lacking, I resorted to reverse engineering Quartus in the attempt to find a solution. So this post is a bit about the transceiver and more about the reverse engineering efforts, which might be relevant in completely different contexts.
I should mention that everything on this page relates to Cyclone 10, even though the output from the tools keep naming different logic elements with “a10″, as if it was Arria 10. Clearly, the transceivers for the two FPGA families are the same.
Software: Quartus Pro 17.1 running on a 64-bit Linux machine (Mint 19).
Cyclone 10′s signal detect is rubbish
The purpose of the signal detector is to tell whether the differential wires are in an electrical idle state, or if there’s some activity on these. This is used by several protocols to wake up the link partners from a low power state: A PCIe link can be awaken by an upstream facing link partner (typically the device waking up the host) from a L2 state by virtue of a beacon, which consists of toggling the polarity at a rate of 30 kHz — 500 MHz. A SATA link can be awaken by one of the link partners transmitting a special data pattern. The USB 3.x protocol also uses out-of-band (OOB) signals if this sort, for various purposes, and calls them LFPS (Low Frequency Pulse Signaling). The toggling rate is defined between 10 — 50 MHz.
The first, relatively simple obstacle, was to turn on the signal detector. The Cyclone 10 GX Transceiver PHY User Guide says in Table 67, regarding rx_std_signaldetect:
When enabled, the signal threshold detection circuitry senses whether the signal level present at the RX input buffer is above the signal detect threshold voltage. You can specify the signal detect threshold using a Quartus Prime Settings File (.qsf) assignment. This signal is required for the PCI Express, SATA and SAS protocols.
Similar notes are made in other places in that guide. However it doesn’t mention that if the transceiver is configured in “basic” mode (as opposed to SATA mode, as well as PCIe mode, I suppose), rx_std_signaldetect is stuck on logic ’1′, so enabling this signal alone isn’t enough.
But the real problem is that the signal detector is probably not good for anything but detecting SATA’s OOB: When I selected SATA mode, I did get some response on rx_std_signaldetect, but it was clearly not detecting the LFPS activity in a useful way. Unlike Cyclone V’s signal detector, Cyclone 10′s detector barely responded at all to a 31.25 MHz LFPS, and the detections occurred with pretty arbitrary timing, often with a pulse when the LFPS signal stopped, and some other random pulses as the wires went into electrical idle. In short, far from the desired assertion when the LFPS signals starts and deassertion when it stops.
Things got better as the toggling frequency increased, and around 125 MHz the assertion of the signal detect was steadily aligned with the onset of the LFPS toggling, however the deassertion was often delayed after the LFPS stopped. So even if the LFPS signal could be guaranteed to be at this frequency (it can’t, as it’s produced by the USB 3.x link partner, and 125 MHz is above maximum) the issue with the deassertion makes it impossible to use it with LFPS, which is extremely sensitive to the timing of onset and release of the toggling.
In fact, it’s probably useless for PCIe as well, as a PCIe beacon is allowed between 30 kHz – 500 MHz. This might explain why recent version of user guides for PCIe block for Cyclone V, Arria V, Cyclone 10 and Arria 10, had this sentence added:
These IP cores also do not support the in-band beacon or sideband WAKE# signal, which are mechanisms to signal a wake-up event to the upstream device.
The problem was probably not spotted for a while because the beacon is rarely used: The PCIe spec utilizes beacon transmission only from a device towards the host (upstream) for the sake of bringing up the link from a low power state. So signal detection by an FPGA for the sake of PCIe is only required when the FPGA acts as a host, and low-power modes are supported. In short, practically never.
What’s left? SATA. That will probably work, because the differential wires toggle rapidly, and it doesn’t matter so much if the detection is a bit off-beat.
So I resorted to detecting the LFPS bursts directly from uncoded received data, rather than using the signal detect feature. The rest of this post relates to my attempts before I gave up.
The options are limited
Quartus is going a long way to be “helpful” by verifying that the parameter assignments make sense with regards to the intended protocol (e.g. SATA, PCIe etc), as reflected by the “prot_mode” parameter. This often means that the fitter throws an error when one tries to alter a parameter from its default. It’s like someone said nope, if you’re using the transceiver for SATA, these and these are the correct analog parameters for the PMA, and if you try otherwise, the fitter will kick your bottom for your own protection.
Or maybe it’s a gentle way of telling us users not to try anything but the protocols for which the transceiver is directly intended for.
The fitter may also ignore assignments because they were assinged an unrelated entity (e.g. to gtx_tx instead of the positive-signal’s name, gtx_txp). So be always sure to look in the fitter report’s “Ignored Assignments” section.
One could speculate that this nanny-like behavior can be disabled by setting one of the pma_*_sup_mode parameters to “engineering_mode” rather than the default “user_mode”, but see below on this.
QSF
I expected to solve this by turning the parameters of the signal detector like I’ve previously done with Cyclone V: By virtue of assignments of QSF file.
So here’s the catch: The User Guide also says that the signal detect threshold can be set by virtue of .qsf assignments, but none such are documented in the it (as of the version for Quartus 20.1), and the Assignment Editor offers no parameter of this sort.
For Cyclone V, it’s documented in the V-Series Transceiver PHY IP Core User Guide around page 20-31, and there are recommended values on this page. My anecdotal experiments seem to indicate that assigning XCVR_* attributes (without the C10 part) to a Cyclone 10 transceiver is accepted however ignored by Quartus. In other words, trying to use Cyclone-V QSF assignments won’t cut.
So let’s start the guesswork on the names of the QSF parameters.
Hint source I: The fitter report
The fitter report has a section called “Receiver Channel”, which shows the attributes of the transceiver’s components as applied de-facto. Among others, there’s a part saying
; -- Name ; frontend_ins|xcvr_inst|xcvr_native_a10_0|xcvr_native_a10_0|g_xcvr_native_insts[0].twentynm_xcvr_native_inst|twentynm_xcvr_native_inst|inst_twentynm_pma|gen_twentynm_hssi_pma_rx_sd.inst_twentynm_hssi_pma_rx_sd ;
; -- Location ; HSSIPMARXSD_1D4 ;
; -- Advanced Parameters ; ;
; -- link ; mr ;
; -- power_mode ; mid_power ;
; -- prot_mode ; sata_rx ;
; -- sd_output_off ; 1 ;
; -- sd_output_on ; 1 ;
; -- sd_pdb ; sd_on ;
; -- sd_threshold ; sdlv_3
It’s actually recommended to go through this part in the fitter report in any case, to make sure it was set up as desired.
But this part allows guessing the names of the parameters for the QSF file. For example, the following assignments are perfectly legal (and match the setting shown above):
set_instance_assignment -name XCVR_C10_RX_SD_OUTPUT_OFF 1 -to gtx_rxp
set_instance_assignment -name XCVR_C10_RX_SD_OUTPUT_ON 1 -to gtx_rxp
set_instance_assignment -name XCVR_C10_RX_SD_THRESHOLD SDLV_3 -to gtx_rxp
It doesn’t take a cyber hacker to see the connection between the QSF parameter names and those appearing in the report. This works for some parameters, and not for other. But this is the easiest way to guess parameter names.
Hint source II: Read the sources
But what’s the allowed values that can be assigned to these parameters? Hints on that can be obtained from the System Verilog files generated for the IP, in particular the one named xcvr_xcvr_native_a10_0_altera_xcvr_native_a10_171_ev4uzpa.sv (the “ev4uzpa” suffix varies), which has a section going:
parameter pma_rx_sd_prot_mode = "basic_rx",//basic_kr_rx basic_rx cpri_rx gpon_rx pcie_gen1_rx pcie_gen2_rx pcie_gen3_rx pcie_gen4_rx qpi_rx sata_rx unused
parameter pma_rx_sd_sd_output_off = 1,//0:28
parameter pma_rx_sd_sd_output_on = 1,//0:15
parameter pma_rx_sd_sd_pdb = "sd_off",//sd_off sd_on
parameter pma_rx_sd_sd_threshold = 3,//0:15
parameter pma_rx_sd_sup_mode = "user_mode",//engineering_mode user_mode
Note the comments, saying which values are allowed for each parameter. On a good day, staying within these value ranges makes the tools accept the assignments, and on an even better day, the fitter won’t throw an error because it considers the values unsuitable.
It’s worth taking a look on the other modules as well, even though they’re likely to have the same comments.
As far as I’ve seen, these parameters are set by the toplevel module for the transceiver IP. QSF assignments, if present, override the former.
Hint source III: What do these assignments mean?
For this I suggest looking at ip/altera/alt_xcvr/alt_xcvr_core/nd/doc/PMA_RegMap.csv (or similar file name) under the root directory of the Quartus installation. Yes, we’re digging in Quartus’ backyard now. I found these files by searching for strings in all files of the Quartus installation. Reverse engineering, after all.
In fact, I’m not sure if this is the correct file to look at, or maybe CR2_PMA_RegMap.csv or whatever. Neither do I know what they mean exactly. It’s however quite evident that these CSV files (opens with your favorite spreadsheet application) were intended to document the register space of the PMA. But the table that shows has a “Attribute Description” column with a few meaningful words on each attribute as well as a column named “Attribute Encoding”, which may happen to be the value to use in a QSF assignment (may and may not work).
There’s also an official register map from Intel available for download, named c10-registermap-official.xlsx, which apparently contains complimentary information. But it’s not possible to deduce QSF names from this file.
Hint source IV: What assignments are legal, then?
I mentioned earlier that the fitter rejects certain value assignment because they apparently don’t make sense. The rules seem to be written as a Tcl script in ip/altera/alt_xcvr/alt_xcvr_core/nd/tcl/ct2_pma_rx_sd_simple.tcl (and similar). Once again, under the Quartus installation root. And yet once again, sometimes this helps, and sometimes it doesn’t.
Hint source V: The names of the QSF paramaters
Up to this point, the names of the parameters to assign in the QSF file were a matter of speculation, based upon similar names in other contexts.
It’s possible to harvest all possible names by searching for strings in one of Quartus’ installed binaries, as shown on this post for non-Pro Quartus 17.1, and this post for Quartus Pro 19.2.
For a complete list of allowed QSF assignment that relate to Cyclone 10 transceivers (or so I groundlessly believe), search for strings in libdb_acf.so, e.g.
$ strings ./quartus/linux64/libdb_acf.so | grep XCVR_C10 | sort
XCVR_C10_CDR_PLL_ANALOG_MODE
XCVR_C10_CDR_PLL_POWER_MODE
XCVR_C10_CDR_PLL_REQUIRES_GT_CAPABLE_CHANNEL
XCVR_C10_CDR_PLL_UC_RO_CAL
XCVR_C10_CMU_FPLL_ANALOG_MODE
XCVR_C10_CMU_FPLL_PLL_DPRIO_CLK_VREG_BOOST
XCVR_C10_CMU_FPLL_PLL_DPRIO_FPLL_VREG1_BOOST
XCVR_C10_CMU_FPLL_PLL_DPRIO_FPLL_VREG_BOOST
XCVR_C10_CMU_FPLL_PLL_DPRIO_STATUS_SELECT
XCVR_C10_CMU_FPLL_POWER_MODE
XCVR_C10_LC_PLL_ANALOG_MODE
XCVR_C10_LC_PLL_POWER_MODE
XCVR_C10_PM_UC_CLKDIV_SEL
XCVR_C10_PM_UC_CLKSEL_CORE
XCVR_C10_PM_UC_CLKSEL_OSC
XCVR_C10_REFCLK_TERM_TRISTATE
XCVR_C10_RX_ADAPT_DFE_CONTROL_SEL
XCVR_C10_RX_ADAPT_DFE_SEL
XCVR_C10_RX_ADAPT_VGA_SEL
XCVR_C10_RX_ADAPT_VREF_SEL
XCVR_C10_RX_ADP_CTLE_ACGAIN_4S
XCVR_C10_RX_ADP_CTLE_EQZ_1S_SEL
XCVR_C10_RX_ADP_DFE_FLTAP_POSITION
XCVR_C10_RX_ADP_DFE_FXTAP1
XCVR_C10_RX_ADP_DFE_FXTAP10
XCVR_C10_RX_ADP_DFE_FXTAP10_SGN
XCVR_C10_RX_ADP_DFE_FXTAP11
XCVR_C10_RX_ADP_DFE_FXTAP11_SGN
XCVR_C10_RX_ADP_DFE_FXTAP2
XCVR_C10_RX_ADP_DFE_FXTAP2_SGN
XCVR_C10_RX_ADP_DFE_FXTAP3
XCVR_C10_RX_ADP_DFE_FXTAP3_SGN
XCVR_C10_RX_ADP_DFE_FXTAP4
XCVR_C10_RX_ADP_DFE_FXTAP4_SGN
XCVR_C10_RX_ADP_DFE_FXTAP5
XCVR_C10_RX_ADP_DFE_FXTAP5_SGN
XCVR_C10_RX_ADP_DFE_FXTAP6
XCVR_C10_RX_ADP_DFE_FXTAP6_SGN
XCVR_C10_RX_ADP_DFE_FXTAP7
XCVR_C10_RX_ADP_DFE_FXTAP7_SGN
XCVR_C10_RX_ADP_DFE_FXTAP8
XCVR_C10_RX_ADP_DFE_FXTAP8_SGN
XCVR_C10_RX_ADP_DFE_FXTAP9
XCVR_C10_RX_ADP_DFE_FXTAP9_SGN
XCVR_C10_RX_ADP_LFEQ_FB_SEL
XCVR_C10_RX_ADP_ONETIME_DFE
XCVR_C10_RX_ADP_VGA_SEL
XCVR_C10_RX_ADP_VREF_SEL
XCVR_C10_RX_BYPASS_EQZ_STAGES_234
XCVR_C10_RX_EQ_BW_SEL
XCVR_C10_RX_EQ_DC_GAIN_TRIM
XCVR_C10_RX_INPUT_VCM_SEL
XCVR_C10_RX_LINK
XCVR_C10_RX_OFFSET_CANCELLATION_CTRL
XCVR_C10_RX_ONE_STAGE_ENABLE
XCVR_C10_RX_POWER_MODE
XCVR_C10_RX_QPI_ENABLE
XCVR_C10_RX_RX_SEL_BIAS_SOURCE
XCVR_C10_RX_SD_OUTPUT_OFF
XCVR_C10_RX_SD_OUTPUT_ON
XCVR_C10_RX_SD_THRESHOLD
XCVR_C10_RX_TERM_SEL
XCVR_C10_RX_TERM_TRI_ENABLE
XCVR_C10_RX_UC_RX_DFE_CAL
XCVR_C10_RX_VCCELA_SUPPLY_VOLTAGE
XCVR_C10_RX_VCM_CURRENT_ADD
XCVR_C10_RX_VCM_SEL
XCVR_C10_RX_XRX_PATH_ANALOG_MODE
XCVR_C10_TX_COMPENSATION_EN
XCVR_C10_TX_DCD_DETECTION_EN
XCVR_C10_TX_DPRIO_CGB_VREG_BOOST
XCVR_C10_TX_LINK
XCVR_C10_TX_LOW_POWER_EN
XCVR_C10_TX_POWER_MODE
XCVR_C10_TX_PRE_EMP_SIGN_1ST_POST_TAP
XCVR_C10_TX_PRE_EMP_SIGN_2ND_POST_TAP
XCVR_C10_TX_PRE_EMP_SIGN_PRE_TAP_1T
XCVR_C10_TX_PRE_EMP_SIGN_PRE_TAP_2T
XCVR_C10_TX_PRE_EMP_SWITCHING_CTRL_1ST_POST_TAP
XCVR_C10_TX_PRE_EMP_SWITCHING_CTRL_2ND_POST_TAP
XCVR_C10_TX_PRE_EMP_SWITCHING_CTRL_PRE_TAP_1T
XCVR_C10_TX_PRE_EMP_SWITCHING_CTRL_PRE_TAP_2T
XCVR_C10_TX_RES_CAL_LOCAL
XCVR_C10_TX_RX_DET
XCVR_C10_TX_RX_DET_OUTPUT_SEL
XCVR_C10_TX_RX_DET_PDB
XCVR_C10_TX_SLEW_RATE_CTRL
XCVR_C10_TX_TERM_CODE
XCVR_C10_TX_TERM_SEL
XCVR_C10_TX_UC_DCD_CAL
XCVR_C10_TX_UC_GEN3
XCVR_C10_TX_UC_GEN4
XCVR_C10_TX_UC_SKEW_CAL
XCVR_C10_TX_UC_TXVOD_CAL
XCVR_C10_TX_UC_TXVOD_CAL_CONT
XCVR_C10_TX_UC_VCC_SETTING
XCVR_C10_TX_USER_FIR_COEFF_CTRL_SEL
XCVR_C10_TX_VOD_OUTPUT_SWING_CTRL
XCVR_C10_TX_XTX_PATH_ANALOG_MODE
So yes, now we’re looking for strings in a binary file.
And yet, all this doesn’t necessarily help
With all these hints, there’s still some pure guesswork. For example, I tried
set_instance_assignment -name XCVR_C10_RX_SD_OUTPUT_ON 3 -to gtx_rxp
and the fitter gave me
Error (15744): The settings must match one or more of these conditions:
Error (15744): ( sup_mode == ENGINEERING_MODE ) OR ( prot_mode != SATA_RX ) OR ( sd_output_on == DATA_PULSE_6 )
Error (15744): But the following assignments violate the above conditions:
Error (15744): sup_mode = USER_MODE
Error (15744): prot_mode = SATA_RX
Error (15744): sd_output_on = DATA_PULSE_10 -- Set by Pin Assignment "XCVR_A10_RX_SD_OUTPUT_ON" (QSF Name "XCVR_A10_RX_SD_OUTPUT_ON")
So first, let’s notice that it blames a XCVR_A10_* assignment, even though I used a XCVR_C10_* assignment in the QSF file. Really.
Also note the hint that setting sup_mode to ENGINEERING_MODE would have let us off the hook. More on that below (however don’t expect much).
But how did the assigning XCVR_C10_RX_SD_OUTPUT_ON with the integer 3 turn into DATA_PULSE_10? Maybe look in PMA_RegMap.csv, mentioned above? But no, DATA_PULSE_10 is assigned 5′b01110 as a value to write to a register, and it’s the 5th value listed, so no matter if you count from zero or one, 3 is not the answer.
Maybe ct2_pma_rx_sd_simple.tcl, also mentioned above? That helps even less, as there’s no sign there that DATA_PULSE_10 would be special. In short, just play with the integer value until hitting gold. Or even better, don’t assign anything, and use the default.
Likewise, setting
set_instance_assignment -name XCVR_C10_RX_SD_OUTPUT_OFF 6 -to gtx_rxp
yields
Error (15744): In atom 'frontend_ins|xcvr_inst|xcvr_native_a10_0|xcvr_native_a10_0|g_xcvr_native_insts[0].twentynm_xcvr_native_inst|twentynm_xcvr_native_inst|inst_twentynm_pma|gen_twentynm_hssi_pma_rx_sd.inst_twentynm_hssi_pma_rx_sd'
Error (15744): The settings must match one or more of these conditions:
Error (15744): ( sup_mode == ENGINEERING_MODE ) OR ( prot_mode != SATA_RX ) OR ( sd_output_off == CLK_DIVRX_2 )
Error (15744): But the following assignments violate the above conditions:
Error (15744): sup_mode = USER_MODE
Error (15744): prot_mode = SATA_RX
Error (15744): sd_output_off = CLK_DIVRX_7 -- Set by Pin Assignment "XCVR_A10_RX_SD_OUTPUT_OFF" (QSF Name "XCVR_A10_RX_SD_OUTPUT_OFF")
Attempting to enable Engineering Mode
Since ENGINEERING_MODE is often mentioned in the fitter’s error messages, I thought maybe enabling it could silence these errors and allow wider options. For example, I attempted to enable the Electrical Idle state on the transmission wires on a non-PCIe transciever by editing one of the files generated by the transceiver IP tools (xcvr_xcvr_native_a10_0.v), changing the line saying
.hssi_tx_pcs_pma_interface_bypass_pma_txelecidle("true"),
to
.hssi_tx_pcs_pma_interface_bypass_pma_txelecidle("false"),
but the fitter threw the following error:
Error (15744): In atom 'xcvr_inst|xcvr_native_a10_0|xcvr_native_a10_0|g_xcvr_native_insts[0].twentynm_xcvr_native_inst|twentynm_xcvr_native_inst|inst_twentynm_pcs|gen_twentynm_hssi_tx_pcs_pma_interface.inst_twentynm_hssi_tx_pcs_pma_interface'
Error (15744): The settings must match one or more of these conditions:
Error (15744): ( sup_mode == ENGINEERING_MODE ) OR ( bypass_pma_txelecidle == TRUE ) OR ( pcie_sub_prot_mode_tx != OTHER_PROT_MODE )
Error (15744): But the following assignments violate the above conditions:
Error (15744): sup_mode = USER_MODE
Error (15744): bypass_pma_txelecidle = FALSE
Error (15744): pcie_sub_prot_mode_tx = OTHER_PROT_MODE
So it tells me that if I want bypass_pma_txelecidle as “false” I have to either set pcie_sub_prot_mode_tx to one of the PCIe modes, or set sup_mode to ENGINEERING_MODE. Changing pcie_sub_prot_mode_tx is out of the question, because the only way to settle the conflicts reported by the fitter is to turn the entire transceiver to follow the predefined PCIe settings. Had I been able to go that path, I would have done that long ago.
So switch to Engineering Mode, whatever that means, by editing the same file, changing
.hssi_tx_pcs_pma_interface_sup_mode("user_mode"),
to
.hssi_tx_pcs_pma_interface_sup_mode("engineering_mode"),
but the fitter really didn’t like that:
Error (15744): In atom 'xcvr_inst|xcvr_native_a10_0|xcvr_native_a10_0|g_xcvr_native_insts[0].twentynm_xcvr_native_inst|twentynm_xcvr_native_inst|inst_twentynm_pcs|gen_twentynm_hssi_tx_pcs_pma_interface.inst_twentynm_hssi_tx_pcs_pma_interface'
Error (15744): The settings must match one or more of these conditions:
Error (15744): ( sup_mode OR ( sup_mode == USER_MODE )
Error (15744): But the following assignments violate the above conditions:
Error (15744): sup_mode = ENGINEERING_MODE
This is somewhat cryptic, because it implies that sup_mode could just evaluate “true” in some way. Anyhow, selecting ENGINEERING_MODE was rejected flat, so that’s not an option for us regular people. There’s probably some secret sauce method to allow this, but that goes beyond what is sensible to work around the tools’ restrictions.
Conclusion
Setting up a Cyclone 10 transceiver for a use other than specifically intended by the FPGA’s vendor is a visit to nomansland. Reverse engineering Quartus does help to some extent, but some issues are left to guessing.
And the transceiver itself appears to be a step backwards compared with the series-V FPGAs. It may reach higher rates, but that came at a cost. Or maybe it’s related to the different silicon process. This way or another, it’s not all that impressive.
Introduction
This post continues my notes on Smart UPS 750, three years later, when it was time to replace the batteries (because they barely held for 13 minutes). It should have been simple, but if I wrote this lengthy post about it, there was clearly something going on. I’ve also written a separate post on general insights on the theory behind lead-acid batteries.
Note that UPSes and their batteries is not my field. These are just my notes as I found my way through. Also, I’ve added to and modified this post several times, so there are definitely inconsistencies in my actions and conclusions, because I learned as I went.
So for short, the main takeaways are these:
- Update the time of last battery replacement with the UPS’ front panel interface (somewhere under Configuration). This makes the UPS realize there are new batteries inside, which changes the way it calculates the estimated runtime.
- Two standard 12V / 7AH lead acid batteries can be used instead of APC’s original battery pack. But check that the terminals are 6mm wide. There are mainly two kinds of terminals: F1 (4.75 mm wide) and F2 (6.35 mm wide).
- The “battery fill” percentage is close to meaningless.
- The displayed battery runtime is not reliable.
- Every now and then, yank the power cord and see how long the UPS lasts. Expect surprises both ways. Battery calibration doesn’t help much.
And now, the deep dive.
Replacing the batteries
The original replacement battery for this UPS is RBC48. But not only is this pack horribly expensive, it’s also hard to find in Israel (or maybe they’re not manufactured anymore?).
The process for battery replacement with non-APC batteries is shown in this video, but it’s really not complicated. Yank off the front panel, then the pull down the metal panel behind the former, and pull out the batteries gently. Use the harness that connects the two existing batteries on the new ones, push them in and you’re done. Plus some packing tape to keep the two batteries together.
However the original batteries’ contact terminals are about 6mm wide (F2), contrary to the ones on the battery I bought (F1), which were considerably smaller. So even though there was no problem connecting the batteries, it wasn’t all that reassuring that the contacts were smaller.
This is a picture taken from above, showing the original pair of batteries I pulled out from the UPS (click to enlarge):
The blue thing in the middle contains a fuse, and the black connector at the top mates with the UPS.
But when I powered up the UPS, the expected runtime shown on the display was just 13 minutes, even though the charge level appeared as 100%. I was surprised to see a 100% charge level on batteries that were just installed, and even more disappointed with the expected runtime. Could it be that bad? Both APC’s runtime chart and my own simple energy calculation (see below) pointed at one hour at least with the load I had. And it didn’t improve after letting the UPS work for a few hours.
My first though was that I had been sold exceptionally junky batteries. But I bought them at a reputable electronics shop, and they carried a timestamp indicating they were fresh (but see below, they were actually junk).
And then it occurred to me that I should tell the UPS that I had replaced batteries. So I went to the part in the UPS’ configuration menu for setting the month and year of the last battery change, and did that. And to my surprise, the runtime was adjusted to 1hr 12 minutes right away. There a few posts out there (this, for example) on how to “reset the battery constant” manually. It seems like this relates to the same thing.
Cute, I thought. But is that figure correct? So I let the UPS run on battery for a while. The estimated runtime went down in pace with the wall clock, but then suddenly, after 23 minutes, it took the power down.
So I reconnected the UPS back to power, and let the battery charge until it reached 100% again. At which point it reported:
$ apcaccess
APC : 001,027,0652
DATE : 2021-08-29 20:21:36 +0300
HOSTNAME : thehost
VERSION : 3.14.14 (31 May 2016) debian
UPSNAME : theups
CABLE : USB Cable
DRIVER : USB UPS Driver
UPSMODE : Stand Alone
STARTTIME: 2021-08-29 18:30:15 +0300
MODEL : Smart-UPS 750
STATUS : ONLINE
BCHARGE : 100.0 Percent
TIMELEFT : 23.0 Minutes
MBATTCHG : 5 Percent
MINTIMEL : 3 Minutes
MAXTIME : 300 Seconds
ALARMDEL : 30 Seconds
BATTV : 26.6 Volts
NUMXFERS : 0
TONBATT : 0 Seconds
CUMONBATT: 0 Seconds
XOFFBATT : N/A
STATFLAG : 0x05000008
MANDATE : 2018-05-22
SERIALNO : AS1821351109
NOMBATTV : 24.0 Volts
FIRMWARE : UPS 09.3 / ID=18
END APC : 2021-08-29 20:22:01 +0300
Smart UPS or what? If the battery died after 23 minutes last time, how much has it left when fully charged? Let me think… 23 minutes!
And yet, that sounds way too short for a new battery. More than 24 hours later, the same runtime estimation remained, going up and down a minute or so occasionally. So that’s that.
It could be correct, however. The way to find out is to try again after a month or so. For that, there’s battery calibration. Which for my UPS means “let the battery drain and measure its way down until it’s empty”. See my follow-up note on that: It’s more or less like unplugging power from the UPS, but less helpful it turns out. Note that the load is said to lose power at the end of this process, even though that didn’t happen to me. So the computer needs to be taken down safely, and then held in a state where a power failure won’t hurt (e.g. stuck in some boot menu). This way, it remains as an electrical load, but nothing bad happens when the power goes down.
Battery calibration is launched from the front panel menu as well. Why the best way to calibrate is to yank the power cord is explained in the follow-up note below.
Make sure the batteries are equally charged
Before connecting a pair of batteries in series (as in my case), it’s a good idea to charge them separately, if possible. That’s true in particular if a discharging test is made immediately after installation. Otherwise, differences in the charging levels may result in a deep discharge of one of the batteries. It’s the same principle as not mixing different batteries in any electronic device.
Alternatively, wait for a while (24 hours?) before the first attempt to discharge the batteries, so both batteries reach the same level by virtue of the floating current.
Otherwise, how can it go wrong? Say, for example, that the UPS stops discharging at 1.7V/cell, which is a rather conservative limit. This means 10.2V for a single battery. But if this is a pair of batteries, this means cutting off at 20.4V. If one battery is still before its steep downhill phase, it can be giving 12.0V. As a result, the other battery will go down to 8.4V when the discharging stops. That’s 1.4V/cell, which isn’t a good idea, except for with a high current discharge. In the datasheets, all discharge curves end at higher than 1.5V/cell, except possibly for with the higher currents.
A note on power consumption
It’s quite obvious that the computer’s power consumption depends on its activity. But as it turns out, the power consumption is higher before Linux’ kernel is loaded.
More specifically, when the computer is on the GRUB menu, the UPS reports 105W / 157VA. One could argue that this is as idle as the computer could be. But then, when Linux’ kernel has been loaded, and it prompts me for my disk unlock password, the power stands at 65W / 120VA. After Linux has booted completely and the desktop is up, it’s 70W / 127VA. So the Linux kernel surely does something to reduce the power consumption when it kicks off.
When compiling a Linux kernel with 12 processes, the power consumption goes to 165W, 217VA.
It’s quite evident that the power factor improves as the power consumption increases. This is in line with Corsair’s promise to attain power level of unity at full capacity (which is 850W, a long way to go).
I run all my battery tests with the computer on GRUB. One has to pick one scenario, and this one represents a computer under moderate load. Whatever that means.
Why a battery drain test is necessary
It’s not clear what my Smart 750 UPS did with the batteries when recharging after they were completely empty. Coulomb Counting is irrelevant on lead acid batteries, because they are constantly discharging. The battery is supplied with a “float current” while being fully charged to keep it in that state.
Besides, there are several factors that influence the battery’s discharging curve. It might discharge nicely for a while, and then suddenly the voltage drops abruptly because the lead plates are worn out. The only way to know about this is to reach that point.
Another factor is that even when the UPS reports that the battery is 100% full, it may still accumulate charge for a long while after that. The UPS might consider the battery full and provide it with a “floating current” but in reality it’s still charging the battery very slowly. A real discharge test should be made no sooner than 24-48 hours of charging.
Follow-up: Recalibrating the battery after two months
Being suspicious about the 23 minutes estimate, I took the computer down after running 76 days (on 14.11.21), kept it powered on so it would load the UPS, and ran a calibration test on the UPS. There’s nothing special about those two and a half months, it just happened to be a convenient time.
During those 76 days, I had monitored the UPS’ answer to apcaccess, and it was steady: The battery voltage remained between 26.8V and 27.0V, and the running time remained around 23 minutes. The UPS didn’t change its mind.
So about the calibration test itself: I started it from the front panel, under the Test Menu, and it read “CalibrationTest in Progress” (the missing space as on the screen). The UPS beeped just like when it’s on battery because of a power loss, and it hummed accordingly. After 20 minutes it went back to main power, claiming to have 15 minutes left. That’s it. It didn’t reach the stage of beeping rapidly, and neither did the power to the computer go off at any time.
At this point I yanked the main power cord, and let it run out. It kept going for another 5 minutes (despite the promise) and then beeped rapidly. 30 seconds later, power went off.
So all in all, it ran for 25 minutes before it died out. The 23 minutes estimate was quite accurate.
Conclusion: Don’t bother calibrating. Just yank the power cord, and measure time.
The third important takeaway is that those “Bull Power” batteries are more like bull-something-else, and I should schedule another battery replacement in a year or so.
And here comes the really funny part. After letting the UPS recharge fully, I checked it again:
$ apcaccess
APC : 001,027,0652
DATE : 2021-11-14 15:19:02 +0200
HOSTNAME : thehost
VERSION : 3.14.14 (31 May 2016) debian
UPSNAME : theups
CABLE : USB Cable
DRIVER : USB UPS Driver
UPSMODE : Stand Alone
STARTTIME: 2021-11-14 13:50:25 +0200
MODEL : Smart-UPS 750
STATUS : ONLINE
BCHARGE : 100.0 Percent
TIMELEFT : 34.0 Minutes
MBATTCHG : 5 Percent
MINTIMEL : 3 Minutes
MAXTIME : 300 Seconds
ALARMDEL : 30 Seconds
BATTV : 26.8 Volts
NUMXFERS : 0
TONBATT : 0 Seconds
CUMONBATT: 0 Seconds
XOFFBATT : N/A
STATFLAG : 0x05000008
MANDATE : 2018-05-22
SERIALNO : AS1821351109
NOMBATTV : 24.0 Volts
FIRMWARE : UPS 09.3 / ID=18
END APC : 2021-11-14 15:19:02 +0200
Is this a smart UPS or what? It just had a calibration cycle which ended with losing power after 25 minutes. How could it estimate 34 minutes now? Maybe if I try again I’ll get 34 minutes? And if this is because the PC consumes less power when Linux is running, why was it only 23 minutes before?
Don’t know. I’ve played with this enough.
Second replacement, 16.4.23
I wasn’t happy with those “bulls” batteries, so I went for an early replacement of these. It turns out that it’s quite difficult to find batteries that are anything but a complete no-name. I barely able to get my hands on a Ritar. And when I say barely, I mean that I managed to get batteries from Afik, which is an Israeli electronics shop (אפיק מ.ל.ה בע”מ in Hebrew). This shop pretends to have its own brand, with a battery labeled AK12-7. But I got a photo of a shipping bill, which indicated that the batteries are manufactured by Ritar. So I went for a pair of those. On the invoice I got from the shop, it says RT1270 (this is the datasheet).
According to the datasheet, the discharge time at 0.55C (see discussion about discharge rates in my separate post) should be one hour. My test ran at 105W, which is grossly 105W/24V =~ 4.4A, which is about 0.63C. So I should have expected 50 minutes or so.
These batteries have small (F1) contact terminals too. I didn’t even think about checking that before I ordered them, actually. But the datasheet says that the battery comes with both F1 and F2 terminals.
I’ll skip to the bottom line right away: These batteries were junk, much worse than the previous ones. It’s plain fraud to call these batteries RT1270. Or maybe it has to do with the timestamp on the battery, which says “201022″. What does it mean? Judging from another battery from the same manufacturer (but different model), which had the timestamp “221215″, the new battery in the UPS was manufactured in October 2020. That could explain things.
So to the story: First I tested the old batteries. I put the computer on the GRUB menu and yanked the power supply from the UPS. Actually, I aborted the first experiment and let the batteries refill on the first attempt. The reason was that the (Western Digital) hard disks made the noise of (proper) activity, so I guess they were doing some kind of self-maintenance or something. I suppose they should be able to take a power loss in the middle of whatever they were doing, but I didn’t want to push my luck. So I just waited long enough, and then they went silent again.
The UPS went down after 9 minutes. Its estimation prior to the test was 12 minutes. The estimation was not bad.
I replaced the batteries with the Afik (Ritar) pair (a.k.a. junk), updated the time of last battery replacement on the UPS, and turned it on. Something was apparently wrong. The UPS was connected to proper power supply, and yet the computer didn’t switch on properly: It powered on, and after a few seconds it had some kind of reboot. It wasn’t clear what the problem was. It could also be because of a problematic USB hub, which tends to irritate the computer (partly because it has its own power supply).
At a later stage I discovered that the BIOS had been reset. Maybe because the computer had been powered off for a relatively long time. How did I notice? Because the CPU fans made more noise than usual, and the CPU’s temperature was lower than usual. I’ve changed the settings in the BIOS to silence the fans somewhat (within safe limits, of course). Plus the RGB sequence of the LEDs inside the computer stopped working. Should I change the BIOS’ battery too?
What I did next was a bit random, but at some point the UPS requested me to confirm that the battery had been replaced.
I took out the battery and pushed it in again. Maybe a loose connection? Who knows. Eventually, I connected the UPS to power, turned on the computer, and all ran fine.
The UPS said that the battery’s charge was 100% and promised all kinds of battery times (mostly around 50 minutes).
I then ran a battery test again. It lasted 5 minutes. When the UPS went on again, it gave the battery 28% and promised 14 minutes. Not so smart UPS.
So I let the battery load to 100% again (that took about 90 minutes), and ran a second battery test.
Bottom line: The UPS promised 49 minutes, in reality it held 11 minutes. This promise made sense, as it was probably based upon 0.63C. No chance in the world that this is an RT1270. Not a decently fresh one, anyhow.
So the old batteries, which I considered new held held 23 minutes when they were new, and 9 minutes after roughly a year and a half. These new batteries held 11 minutes as new!
Restarting the UPS, it reported 0% battery (make sense, doesn’t it?) and no 0 minutes runtime. After getting fully charged, the UPS promised 17 minutes. But that was with Linux running, so the power consumption is lower than the test. So it’s a reasonable estimation for a horrible battery.
Conclusion: I need to replace the batteries again, within a year. The computer shuts itself down after 5 minutes, so I guess it will be fine, but this is ridicolous.
Strategy for the next time
After two fiascoes, I’m changing strategy. So these are things I’m going to insist on:
- The marking on the battery should belong to a company that has a more or less proper website, from which I can download a datasheet.
- The datasheet should include discharging characteristics, either in the form of curves or tables, preferably both.
- The battery’s model code, as it appears in the datasheet, should be printed on the battery.
- If the battery is marked with the name of an Israeli importer, it’s by far not as good, even if the importer publishes a datasheet that is possibly identical to the one of the manufacturer.
- The manufacturing date of the battery should be marked on the battery. It should be easy to figure out what it means, and the battery should be fresh of course.
- Prefer an F2 terminal
- Reputable brand is a bonus.
The rationale: A datasheet with discharging characteristics is a commitment. If a large company buys 1000 batteries, and tests a few samples just to find out that they don’t meet the specs, a hefty lawsuit may follow. No sane battery manufacturer will take that risk if it can’t ensure that their batteries perform well.
I couldn’t find such a datasheet for “Bulls Power”, and surely not for that Afik piece of junk. The latter was probably manufactured by Ritar, but not with their commitment.
Not going to insist on: A known brand. Ritar batteries are unavailable in Israel at the moment, an Yuasa’s batteries cost a fortune. Their datasheet shows discharging graphs, of course. But no tables.
But more than anything, it seems like it’s a matter of age. I don’t think anyone manufactures really bad batteries to begin with. I speculate that those junk brands buy old batteries from good manufacturers, and market them with a different name, at a low price.
Who to buy from
Obviously, web shops are the easiest options. If they cooperate with sending me photos of the battery (see below). If not, go for bricks and mortar electricity shops (Erco, for example). Also consider shops and repair shops for motored toy vehicles for small kids. The search word in Hebrew for that is ממונעים (which means “motored”) but is somehow a collective term for these toys.
Is this a Covid-19 thing?
I called up Lion Electronic‘s sales (a company I consider reputable) for a couple of Yuasa batteries (200 NIS each, April 2023), and I was told that they won’t sell me two batteries at that price, because they are imported specially, and that has an extra cost.
My next hope was to get an Aokly 6FM7/6FM9, which are quite available in Israel. However, as of April 2023, two different suppliers told me that it had no manufacturing date printed on it. Which is really weird. So this manufacturer is irrelevant.
I made other significant attempts to get a battery that meets the above criteria in May 2023, and failed: Every time I asked for a photo of the battery and its timestamp, I got no response, or a rather negative one. I got online purchase orders canceled twice while attempting to buy a CSB battery from AlPesek, which is an importer of this battery. After talking with someone at the company about manufacturing date, and putting that request in the comments of the order, the company canceled the order, claiming that they don’t work with end customers (so why was it listed on the website?) and the batteries were delisted from the website. Trying to get the same battery from a web shop (that is, from the same source, but indirectly), I got the order canceled again, with the supposed reason that the minimum purchase for resellers is 10 units.
And there were several other attempts that ended with the other side simply not cooperating with sending me a couple of photos.
At one of Erco‘s shops, I found a VisionNet battery, which is apparently a local brand made up by Telran (a respectful company). The interesting thing about this battery is that it said “Production date 06/2022″ on the battery itself, and the guy at the shop told me that it had arrived a week earlier (and I believe him, the boxes in the storage were new). Also, there are several photos of the battery with this model, but they look different, and none have that timestamp. So it appears like Telran shops around for batteries and put their own marking on them. And that leaves the question how a battery that arrived a week ago is a year old. So the production date is probably something that the real manufacturer promised. Can that be trusted? I don’t know. If there was any battery I thought seriously about trying, it’s a pair of these.
Given this rather odd picture, I speculate that there was a lot of batteries that got stuck in storage, somehow related to Covid-19. And now the manufacturers need to get rid of a whole lot of old, and hence damaged batteries. That’s the best explanation I have to all these batteries with no date marking, plus a lot of weird behavior.
My decision was hence to stop my attempts to replaced the batteries in my UPS for six months, and try again somewhere towards the end of 2023.
Third battery replacement, 17.12.23
I went into Erco‘s shop in Kiryat Ata, and was offered a couple of Vega Power NP12-7 batteries. Never heard about that brand? Neither have I. Neither did this company have a reassuring Internet footprint. But the batteries had an an engraved timestamp saying 10/08/2023 and they had F2 terminals. So I bought a couple of them, at a price of 120 NIS, VAT included. Cheap, in other words.
And the answer to my previous question: Yes, the lack of fresh batteries was most likely a Covid-19 thing.
I ran a detailed discharging test on these two batteries (as well as on the Afik batteries), which I write about in that separate post. Spoiler: The discharging test doesn’t say too much about how the batteries will behave inside the UPS.
First, I wanted to check the situation of the Afik batteries that were already inside the UPS. I brought down the computer to where GRUB waits for prompt, so the UPS reported power consumption was 105W / 150VA. At this point, the UPS promised 12 minutes.
So I yanked the power cord. The UPS began panic beeping after 7:30 minutes, stating 3 minutes were left, which dropped down to 0% and 0 minutes quite soon. Contrary to its pessimistic estimation, the UPS kept up the power for quite long after that, so the total time with power ended up at 22:00 minutes.
Say what? The same batteries held 11 minutes when I first put them in the UPS, and now the time doubled?!
If I don’t want to develop mysterious theories about what happens inside the battery, the explanation could be that the battery was far from being fully charged when I tested it a few months earlier: The UPS claimed that the battery was 100% full after 90 minutes, but it’s quite possible that this estimation was anything but accurate.
It’s also worth citing Power Sonic’s Technical manual: “By cycling the battery a few times or float charging it for a month or two, the highest level of capacity development is achieved. Power-Sonic batteries are fully charged before leaving the factory, but full capacity is realized only after the battery has been cycled a few times or been on float charge for some time”. This is written about another lead-acid battery of course, but maybe it’s a thing that a few months inside the UPS does the battery good.
I replaced the Afik batteries with Vega Power’s. These batteries had been separately charged with another UPS for 48 hours each. After updating the battery replacement date on the UPS (i.e. telling the UPS that there are new batteries inside), the UPS initially promised an uptime of above an hour, and then swayed along until it gradually stabilized at 49 minutes.
Yanked the power cord. All was good and calm until panic beeping started after 28:15 minutes, after which the power went out just a few seconds later. Boom.
The computer was stable on the GRUB menu throughout all this.
So what’s the verdict? I don’t know. The discharging tests that I ran on both pairs of batteries made things even more confusing: Vega Power performed really well, and Afik didn’t worse, but not much worse.
The only clear conclusion is that a UPS discharge test is required every now and then. There’s no way around this.
Repeated discharging test, 22.3.24
Three months later, I ran the same discharging test on the Vega Power batteries. The panic beep began after 22:10 minutes. At 31:30 minutes, the panic beep went off, and then it went on and off sporadically as the UPS’ estimation for remaining time fluctuated between 3 and 6 minutes. At 33:50 minutes the panic beep became steadily on again, and at 34:20 the UPS went off.
In short: Three months of floating current added 6 minutes to the battery time. And an even more confused ride as the battery discharged.
Introduction
Occasionally, I download / upload huge files, and it kills my internet connection for plain browsing. I don’t want to halt the download or suspend it, but merely calm it down a bit, temporarily, for doing other stuff. And then let it hog as much as it want again.
There are many ways to do this, and I went for firejail. I suggest reading this post of mine as well on this tool.
Firejail gives you a shell prompt, which runs inside a mini-container, like those cheap virtual hosting services. Then run wget or youtube-dl as you wish from that shell.
It has practically access to everything on the computer, but the network interface is controlled. Since firejail is based on cgroups, all processes and subprocesses are collectively subject to the network bandwidth limit.
Using firejail requires setting up a bridge network interface. This is a bit of container hocus-pocus, and is necessary to get control over the network data flow. But it’s simple, and it can be done once (until the next reboot, unless the bridge is configured permanently, something I don’t bother).
Setting up a bridge interface
Remember: Do this once, and just don’t remove the interface when done with it.
You might need to
# apt install bridge-utils
So first, set up a new bridge device (as root):
# brctl addbr hog0
and give it an IP address that doesn’t collide with anything else on the system. Otherwise, it really doesn’t matter which:
# ifconfig hog0 10.22.1.1/24
What’s going to happen is that there will be a network interface named eth0 inside the container, which will behave as if it was connected to a real Ethernet card named hog0 on the computer. Hence the container has access to everything that is covered by the routing table (by means of IP forwarding), and is also subject to the firewall rules. With my specific firewall setting, it prevents some access, but ppp0 isn’t blocked, so who cares.
To remove the bridge (no real reason to do it):
# brctl delbr hog0
Running the container
Launch a shell with firejail (I called it “nethog” in this example):
$ firejail --net=hog0 --noprofile --name=nethog
This starts a new shell, for which the bandwidth limit is applied. Run wget or whatever from here.
Note that despite the –noprofile flag, there are still some directories that are read-only and some are temporary as well. It’s done in a sensible way, though so odds are that it won’t cause any issues. Running “df” inside the container gives an idea on what is mounted how, and it’s scarier than the actual situation.
But be sure to check that the files that are downloaded are visible outside the container.
From another shell prompt, outside the container go something like (doesn’t require root):
$ firejail --bandwidth=nethog set hog0 800 75
Removing bandwith limit
Configuring interface eth0
Download speed 6400kbps
Upload speed 600kbps
cleaning limits
configuring tc ingress
configuring tc egress
To drop the bandwidth limit:
$ firejail --bandwidth=nethog clear hog0
And get the status (saying, among others, how many packets have been dropped):
$ firejail --bandwidth=nethog status
Notes:
- The “eth0″ mentioned in firejail’s output blob relates to the interface name inside the container. So the “real” eth0 remains untouched.
- Actual download speed is slightly slower.
- The existing group can be joined by new processes with firejail –join, as well as from firetools.
- Several containers may use the same bridge (hog0 in the example above), in which case each has its own independent bandwidth setting. Note that the commands configuring the bandwidth limits mention both the container’s name and the bridge.
Working with browsers
When starting a browser from within a container, pay attention to whether it really started a new process. Using firetools can help.
If Google Chrome says “Created new window in existing browser session”, it didn’t start a new process inside the container, in which case the window isn’t subject to bandwidth limitation.
So close all windows of Chrome before kicking off a new one. Alternatively, this can we worked around by starting the container with.
$ firejail --net=hog0 --noprofile --private --name=nethog
The –private flags creates, among others, a new volatile home directory, so Chrome doesn’t detect that it’s already running. Because I use some other disk mounts for the large partitions on my computer, it’s still possible to download stuff to them from within the container.
But extra care is required with this, and regardless, the new browser doesn’t remember passwords and such from the private container.
Using a different version of Google Chrome
This isn’t really related, and yet: What if I want to use a different version of Chrome momentarily, without upgrading? This can be done by downloading the .deb package, and extracting its files as shown on this post. Then copy the directory opt/google/chrome in the package’s “data” files to somewhere reachable by the jail (e.g. /bulk/transient/google-chrome-105.0/).
All that is left is to start a jail with the –private option as shown above (possibly without the –net flag, if throttling isn’t required) and go e.g.
$ /bulk/transient/google-chrome-105.0/chrome &
So the new browser can run while there are still windows of the old one open. The advantage and disadvantage of jailing is that there’s no access to the persistent data. So the new browser doesn’t remember passwords. This is also an advantage, because there’s a chance that the new version will mess up things for the old version.
This should have been a trivial task, but it turned out quite difficult. So these are my notes for the next time. Octave 4.2.2 under Linux Mint 19, using qt5ct plugin with GNU plot (or else I get blank plots).
So this is the small function I wrote for creating a plot and a thumbnail:
function []=toimg(fname, alt)
grid on;
saveas(gcf, sprintf('%s.png', fname), 'png');
print(gcf, sprintf('%s_thumb.png', fname), '-dpng', '-color', '-S280,210');
disp(sprintf('<a href="/media/%s.png" target="_blank"><img alt="%s" src="/media/%s_thumb.png" style="width: 280px; height: 210px;"></a>', fname, alt, fname));
The @alt argument becomes the image’s alternative text when shown on the web page.
The call to saveas() creates a 1200x900 image, and the print() call creates a 280x210 one (as specified directly). I take it that print() will create a 1200x900 without any specific argument for the size, but I left both methods, since this is how I ended up after struggling, and it’s better to have both possibilities shown.
To add some extra annoyment, toimg() always plots the current figure, which is typically the last figure plotted. Which is not necessarily the figure that has focus. As a matter of fact, even if the current figure is closed by clicking the upper-right X, it remains the current figure. Calling toimg() will make it reappear and get plotted. Which is really weird behavior.
The apparently only way around this is to use figure() to select the desired current figure before calling ioimg(), e.g.
>> figure(4);
The good news is that the figure numbers match those appearing on the windows’ titles. This also explains why the numbering doesn’t reset when closing all figure windows manually. To really clear all figures, go
>> close all hidden
Other oddities
- ginput() simply doesn’t work. The workaround is to double-click any point (with left button) and the coordinates of this point are copied into the clipboard. Paste it anywhere. Odd, but not all that bad.
- Zooming in with right-click and then left-click doesn’t affect axis(). As a result, saving the plot as an image is not affected by this zoom feature. Wonky workaround: Use the double-click trick above to obtain the coordinates of relevant corners, and use axis() to set them properly. Bonus: One gets the chance to adjust the figures for a sleek plot. If anyone knows how to save a plot as it’s shown by zooming, please comment below.
So I have written a function, showfile() for Octave 4.2.2 on Linux, which accepts a file name as its argument. And now I want to run it on all files in the current directory that match a certain pattern. How?
So first, obtain the list of files, and put it in a variable:
>> x=ls('myfiles*.dat');
This creates a matrix of chars, with each row containing the name of one file. The number of columns of this matrix is the length of longest file name, with the other rows padded with spaces (yes, ASCII 0x20).
So to call the function on all files:
>> for i=1:rows(x) ; showfile(strtrim(x(i,:))); end
The call to strtrim() removes the trailing spaces (those that were padded), so that the argument is the actual file name. If the real file name contains leading or trailing spaces, this won’t work (but who does that?). Spaces in the middle of the file name are OK, as strtrim() doesn’t touch them.
After a few days being happy with not getting spam, I started to suspect that something is completely wrong with receiving mail. As I’m using fetchmail to get mail from my own server running dovecot v2.2.13, I’m used to getting notifications when fetchmail is unhappy. But there was no such.
Checking up the server’s logs, there were tons of these messages:
dovecot: master: Warning: service(pop3-login): process_limit (100) reached, client connections are being dropped
Restarting dovecot got it back running properly again, and I got a flood of the mails that were pending on the server. This was exceptionally nasty, because mails stopped arriving silently.
So what was the problem? The clue is in these log messages, which occurred about a minute after the system’s boot (it’s a VPS virtual machine):
Jul 13 11:21:46 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process
Jul 13 11:21:46 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process
Jul 13 11:21:46 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process
Jul 13 11:21:46 dovecot: master: Error: service(log): child 1210 killed with signal 9
These three services are helper processes for dovecot, as can be seen in the output of systemctl status:
├─dovecot.service
│ ├─11690 /usr/sbin/dovecot -F
│ ├─11693 dovecot/anvil
│ ├─11694 dovecot/log
│ ├─26494 dovecot/config
│ ├─26495 dovecot/auth
│ └─26530 dovecot/auth -w
What seems to have happened is that these processes failed to launch properly within the 30 second timeout limit, and were therefore killed by dovecot. And then attempts to make pop3 connections seem to have got stuck, with the forked processes that are made for each connection remaining. Eventually, they reached the maximum of 100.
The reason this happened only now is probably that the hosting server had some technical failure and was brought down for maintenance. When it went up again, all VMs were booted at the same time, so they were all very slow in the beginning. Hence it took exceptionally long to kick off those helper processes. The 30 seconds timeout kicked in.
The solution? Restart dovecot once in 24 hours with a plain cronjob. Ugly, but works. In the worst case, mail will be delayed for 24 hours. This is a very rare event to begin with.
Introduction
Checking Xillybus’ bundle for Kintex Ultrascale+ on Vivado 2020.1, I got several critical warnings related to the PCIe block. As the bundle is intended to show how Xillybus’ IP core is used for simplifying communication with the host, these warnings aren’t directly related, and yet they’re unacceptable.
This bundle is designed to work with Vivado 2017.3 and later: It sets up the project by virtue of a Tcl script, which among others calls the upgrade_ip function for updating all IPs. Unfortunately, a bug in Vivado 2020.1 (and possibly other versions) causes the upgraded PCIe block to end up misconfigured.
This bug applies to Zynq Ultrascale+ as well, but curiously enough not with Virtex Ultrascale+. At least with my setting there was no problem.
The problem
Having upgraded an UltraScale+ Integrated Block (PCIE4) for PCI Express IP block from Vivado 2017.3 (or 2018.3) to Vivado 2020.1, I got several Critical Warnings. Three during synthesis:
[Vivado 12-4739] create_clock:No valid object(s) found for '-objects [get_pins -filter REF_PIN_NAME=~TXOUTCLK -of_objects [get_cells -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GT*E4_CHANNEL_PRIM_INST}]]'. ["project/pcie_ip_block/source/ip_pcie4_uscale_plus_x0y0.xdc":127]
[Vivado 12-4739] get_clocks:No valid object(s) found for '--of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]'. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":63]
[Vivado 12-4739] get_clocks:No valid object(s) found for '--of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]'. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":64]
and another seven during implementation:
[Vivado 12-4739] create_clock:No valid object(s) found for '-objects [get_pins -filter REF_PIN_NAME=~TXOUTCLK -of_objects [get_cells -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GT*E4_CHANNEL_PRIM_INST}]]'. ["project/pcie_ip_block/source/ip_pcie4_uscale_plus_x0y0.xdc":127]
[Vivado 12-4739] set_clock_groups:No valid object(s) found for '-group [get_clocks -of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]]'. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":63]
[Vivado 12-4739] set_clock_groups:No valid object(s) found for '-group '. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":63]
[Vivado 12-4739] set_clock_groups:No valid object(s) found for '-group [get_clocks -of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]]'. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":64]
[Vivado 12-4739] set_clock_groups:No valid object(s) found for '-group '. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":64]
[Vivado 12-5201] set_clock_groups: cannot set the clock group when only one non-empty group remains. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":63]
[Vivado 12-5201] set_clock_groups: cannot set the clock group when only one non-empty group remains. ["project/pcie_ip_block/synth/pcie_ip_block_late.xdc":64]
The first warning in each group points at this line in ip_pcie4_uscale_plus_x0y0.xdc, which was automatically generated by the tools:
create_clock -period 4.0 [get_pins -filter {REF_PIN_NAME=~TXOUTCLK} -of_objects [get_cells -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GT*E4_CHANNEL_PRIM_INST}]]
And the other at these two lines in pcie_ip_block_late.xdc, also generated by the tools:
set_clock_groups -asynchronous -group [get_clocks -of_objects [get_ports sys_clk]] -group [get_clocks -of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]]
set_clock_groups -asynchronous -group [get_clocks -of_objects [get_pins -hierarchical -filter {NAME =~ *gen_channel_container[1200].*gen_gtye4_channel_inst[3].GTYE4_CHANNEL_PRIM_INST/TXOUTCLK}]] -group [get_clocks -of_objects [get_ports sys_clk]]
So this is clearly about a reference to a non-existent logic cell supposedly named gen_channel_container[1200], and in particular that index, 1200, looks suspicious.
I would have been relatively fine with ignoring these warnings had it been just the set_clock_groups that failed, as these create false paths. If the design implements properly without these, it’s fine. But failing a create_clock command is serious, as this can leave paths unconstrained. I’m not sure if this is indeed the case, and it doesn’t matter all that much. One shouldn’t get used to ignoring critical warnings.
Looking at the .xci file for this PCIe block, it’s apparent that several changes were made to it while upgrading to 2020.1. Among those changes, these three lines were added:
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT">GTHE4_CHANNEL_X49Y99</spirit:configurableElementValue>
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT_CONTAINER">1200</spirit:configurableElementValue>
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT_QUAD_INX">3</spirit:configurableElementValue>
Also, somewhere else in the XCI file, this line was added:
<spirit:configurableElementValue spirit:referenceId="PARAM_VALUE.MASTER_GT">GTHE4_CHANNEL_X49Y99</spirit:configurableElementValue>
So there’s a bug in the upgrading mechanism, which sets some internal parameter to select the a nonexistent GT site.
The manual fix (GUI)
To rectify the wrong settings manually, enter the settings of the PCIe block, and click the checkbox for “Enable GT Quad Selection” twice: Once for unchecking, and once for checking it. Make sure that the selected GT hasn’t changed.
Then it might be required to return some unrelated settings to their desired values. In particular, the PCI Device ID and similar attributes change to Xilinx’ default as a result of this. It’s therefore recommended to make a copy of the XCI file before making this change, and then use a diff tool to compare the before and after files, looking for irrelevant changes. Given that this revert to default has been going on for so many years, it seems like Xilinx considers this a feature.
But this didn’t solve my problem, as the bundle needs to set itself correctly out of the box.
Modifying the XCI file? (Not)
The immediate thing to check was whether this problem applies to PCIe blocks that are created in Vivado 2020.1 from scratch inside a project which is set to target KCU116 (which is what the said Xillybus bundle targets). As expected, it doesn’t — this occurs just on upgraded IP blocks: With the project that was set up from scratch, the related lines in the XCI file read:
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT">GTYE4_CHANNEL_X0Y7</spirit:configurableElementValue>
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT_CONTAINER">1</spirit:configurableElementValue>
<spirit:configurableElementValue spirit:referenceId="MODELPARAM_VALUE.MASTER_GT_QUAD_INX">3</spirit:configurableElementValue>
and
<spirit:configurableElementValue spirit:referenceId="PARAM_VALUE.MASTER_GT">GTYE4_CHANNEL_X0Y7</spirit:configurableElementValue>
respectively. These are values that make sense.
With this information at hand, my first attempt to solve this was to add the four new lines to the old XCI file. This allowed using the XCI file with Vivado 2020.1 properly, however synthesizing the PCIe block on older Vivado versions failed: As it turns out, all MODELPARAM_VALUE attributes become instantiation parameters for pcie_uplus_pcie4_uscale_core_top inside the PCIe block. However looking at the source file (on 2020.1), these parameters are indeed defined (only in those generated in 2020.1), and yet they are unused, like many other instantiation parameters in this module. So apparently, Vivado’s machinery generates an instantiation parameter for each of these, even if they’re not used. Those unused parameters are most likely intended for scripting.
So this trick made Vivado instantiate the pcie_uplus_pcie4_uscale_core_top with instantiation parameters that it doesn’t have, and hence its synthesis failed. Dead end.
I didn’t examine the possibility to deselect “Enable GT Quad Selection” in the original block, because Vivado 2017.3 chooses the wrong GT for the board without this option.
Workaround with Tcl
Eventually, I solved the problem by adding a few lines to the Tcl script.
Assuming that $ip_name has been set to the name of the PCIe block IP, this Tcl snippet rectifies the bug:
if {![string equal "" [get_property -quiet CONFIG.MASTER_GT [get_ips $ip_name]]]} {
set_property -dict [list CONFIG.en_gt_selection {true} CONFIG.MASTER_GT {GTYE4_CHANNEL_X0Y7}] [get_ips $ip_name]
}
This snippet should of course be inserted after updating the IP core (with e.g. upgrade_ip [get_ips]). The code first checks if the MASTER_GT is defined, and only if so, it sets it to the desired value. This ensures that nothing happens with the older Vivado versions. Note the “quiet” flag of get_properly, which prevents it from generating an error if the property isn’t defined. Rather, it returns an empty string if that’s the case, which is what the result is compared against.
Setting MASTER_GT this way also rectifies GT_CONTAINER correctly, and surprisingly enough, this doesn’t change anything it shouldn’t, and in particular, the Device IDs remain intact.
However the disadvantage with this solution is that the GT to select is hardcoded in the Tcl code. But that’s fine in my case, for which a specific board (KCU116) is targeted by the bundle.
Another way to go, which is less recommended, is to emulate the check and uncheck of “Enable GT Quad Selection”:
if {![string equal "" [get_property -quiet CONFIG.MASTER_GT [get_ips $ip_name]]]} {
set_property CONFIG.en_gt_selection {false} [get_ips $ip_name]
set_property CONFIG.en_gt_selection {true} [get_ips $ip_name]
}
However turning the en_gt_selection flag off and on again also resets the Device ID to default as with manual toggling of the checkbox. And even though it set the MASTER_GT correctly in my specific case, I’m not sure whether this can be relied upon.
