APC Smart UPS 750 battery replacement notes

Introduction

This post continues my notes on Smart UPS 750, three years later, when it was time to replace the batteries (because they barely held for 13 minutes). It should have been simple, but if I wrote this lengthy post about it, there was clearly something going on. I’ve also written a separate post on general insights on the theory behind lead-acid batteries.

Note that UPSes and their batteries is not my field. These are just my notes as I found my way through. Also, I’ve added to and modified this post several times, so there are definitely inconsistencies in my actions and conclusions, because I learned as I went.

So for short, the main takeaways are these:

• Update the time of last battery replacement with the UPS’ front panel interface (somewhere under Configuration). This makes the UPS realize there are new batteries inside, which changes the way it calculates the estimated runtime.
• Two standard 12V / 7AH lead acid batteries can be used instead of APC’s original battery pack. But check that the terminals are 6mm wide. There are mainly two kinds of terminals: F1 (4.75 mm wide) and F2 (6.35 mm wide).
• The “battery fill” percentage is close to meaningless.
• The displayed battery runtime is not reliable.
• Every now and then, yank the power cord and see how long the UPS lasts. Expect surprises both ways. Battery calibration doesn’t help much.

And now, the deep dive.

Replacing the batteries

The original replacement battery for this UPS is RBC48. But not only is this pack horribly expensive, it’s also hard to find in Israel (or maybe they’re not manufactured anymore?).

The process for battery replacement with non-APC batteries is shown in this video, but it’s really not complicated. Yank off the front panel, then the pull down the metal panel behind the former, and pull out the batteries gently. Use the harness that connects the two existing batteries on the new ones, push them in and you’re done. Plus some packing tape to keep the two batteries together.

However the original batteries’ contact terminals are about 6mm wide (F2), contrary to the ones on the battery I bought (F1), which were considerably smaller. So even though there was no problem connecting the batteries, it wasn’t all that reassuring that the contacts were smaller.

This is a picture taken from above, showing the original pair of batteries I pulled out from the UPS (click to enlarge):

The blue thing in the middle contains a fuse, and the black connector at the top mates with the UPS.

But when I powered up the UPS, the expected runtime shown on the display was just 13 minutes, even though the charge level appeared as 100%. I was surprised to see a 100% charge level on batteries that were just installed, and even more disappointed with the expected runtime. Could it be that bad? Both APC’s runtime chart and my own simple energy calculation (see below) pointed at one hour at least with the load I had. And it didn’t improve after letting the UPS work for a few hours.

My first though was that I had been sold exceptionally junky batteries. But I bought them at a reputable electronics shop, and they carried a timestamp indicating they were fresh (but see below, they were actually junk).

And then it occurred to me that I should tell the UPS that I had replaced batteries. So I went to the part in the UPS’ configuration menu for setting the month and year of the last battery change, and did that. And to my surprise, the runtime was adjusted to 1hr 12 minutes right away. There a few posts out there (this, for example) on how to “reset the battery constant” manually. It seems like this relates to the same thing.

Cute, I thought. But is that figure correct? So I let the UPS run on battery for a while. The estimated runtime went down in pace with the wall clock, but then suddenly, after 23 minutes, it took the power down.

So I reconnected the UPS back to power, and let the battery charge until it reached 100% again. At which point it reported:

$ apcaccess
APC      : 001,027,0652
DATE     : 2021-08-29 20:21:36 +0300
HOSTNAME : thehost
VERSION  : 3.14.14 (31 May 2016) debian
UPSNAME  : theups
CABLE    : USB Cable
DRIVER   : USB UPS Driver
UPSMODE  : Stand Alone
STARTTIME: 2021-08-29 18:30:15 +0300
MODEL    : Smart-UPS 750
STATUS   : ONLINE
BCHARGE  : 100.0 Percent
TIMELEFT : 23.0 Minutes
MBATTCHG : 5 Percent
MINTIMEL : 3 Minutes
MAXTIME  : 300 Seconds
ALARMDEL : 30 Seconds
BATTV    : 26.6 Volts
NUMXFERS : 0
TONBATT  : 0 Seconds
CUMONBATT: 0 Seconds
XOFFBATT : N/A
STATFLAG : 0x05000008
MANDATE  : 2018-05-22
SERIALNO : AS1821351109
NOMBATTV : 24.0 Volts
FIRMWARE : UPS 09.3 / ID=18
END APC  : 2021-08-29 20:22:01 +0300

Smart UPS or what? If the battery died after 23 minutes last time, how much has it left when fully charged? Let me think… 23 minutes!

And yet, that sounds way too short for a new battery. More than 24 hours later, the same runtime estimation remained, going up and down a minute or so occasionally. So that’s that.

It could be correct, however. The way to find out is to try again after a month or so. For that, there’s battery calibration. Which for my UPS means “let the battery drain and measure its way down until it’s empty”. See my follow-up note on that: It’s more or less like unplugging power from the UPS, but less helpful it turns out. Note that the load is said to lose power at the end of this process, even though that didn’t happen to me. So the computer needs to be taken down safely, and then held in a state where a power failure won’t hurt (e.g. stuck in some boot menu). This way, it remains as an electrical load, but nothing bad happens when the power goes down.

Battery calibration is launched from the front panel menu as well. Why the best way to calibrate is to yank the power cord is explained in the follow-up note below.

Make sure the batteries are equally charged

Before connecting a pair of batteries in series (as in my case), it’s a good idea to charge them separately, if possible. That’s true in particular if a discharging test is made immediately after installation. Otherwise, differences in the charging levels may result in a deep discharge of one of the batteries. It’s the same principle as not mixing different batteries in any electronic device.

Alternatively, wait for a while (24 hours?) before the first attempt to discharge the batteries, so both batteries reach the same level by virtue of the floating current.

Otherwise, how can it go wrong? Say, for example, that the UPS stops discharging at 1.7V/cell, which is a rather conservative limit. This means 10.2V for a single battery. But if this is a pair of batteries, this means cutting off at 20.4V. If one battery is still before its steep downhill phase, it can be giving 12.0V. As a result, the other battery will go down to 8.4V when the discharging stops. That’s 1.4V/cell, which isn’t a good idea, except for with a high current discharge. In the datasheets, all discharge curves end at higher than 1.5V/cell, except possibly for with the higher currents.

A note on power consumption

It’s quite obvious that the computer’s power consumption depends on its activity. But as it turns out, the power consumption is higher before Linux’ kernel is loaded.

More specifically, when the computer is on the GRUB menu, the UPS reports 105W / 157VA. One could argue that this is as idle as the computer could be. But then, when Linux’ kernel has been loaded, and it prompts me for my disk unlock password, the power stands at 65W / 120VA. After Linux has booted completely and the desktop is up, it’s 70W / 127VA. So the Linux kernel surely does something to reduce the power consumption when it kicks off.

When compiling a Linux kernel with 12 processes, the power consumption goes to 165W, 217VA.

It’s quite evident that the power factor improves as the power consumption increases. This is in line with Corsair’s promise to attain power level of unity at full capacity (which is 850W, a long way to go).

I run all my battery tests with the computer on GRUB. One has to pick one scenario, and this one represents a computer under moderate load. Whatever that means.

Why a battery drain test is necessary

It’s not clear what my Smart 750 UPS did with the batteries when recharging after they were completely empty. Coulomb Counting is irrelevant on lead acid batteries, because they are constantly discharging. The battery is supplied with a “float current” while being fully charged to keep it in that state.

Besides, there are several factors that influence the battery’s discharging curve. It might discharge nicely for a while, and then suddenly the voltage drops abruptly because the lead plates are worn out. The only way to know about this is to reach that point.

Another factor is that even when the UPS reports that the battery is 100% full, it may still accumulate charge for a long while after that. The UPS might consider the battery full and provide it with a “floating current” but in reality it’s still charging the battery very slowly. A real discharge test should be made no sooner than 24-48 hours of charging.

Follow-up: Recalibrating the battery after two months

Being suspicious about the 23 minutes estimate, I took the computer down after running 76 days (on 14.11.21), kept it powered on so it would load the UPS, and ran a calibration test on the UPS. There’s nothing special about those two and a half months, it just happened to be a convenient time.

During those 76 days, I had monitored the UPS’ answer to apcaccess, and it was steady: The battery voltage remained between 26.8V and 27.0V, and the running time remained around 23 minutes. The UPS didn’t change its mind.

So about the calibration test itself: I started it from the front panel, under the Test Menu, and it read “CalibrationTest in Progress” (the missing space as on the screen). The UPS beeped just like when it’s on battery because of a power loss, and it hummed accordingly. After 20 minutes it went back to main power, claiming to have 15 minutes left. That’s it. It didn’t reach the stage of beeping rapidly, and neither did the power to the computer go off at any time.

At this point I yanked the main power cord, and let it run out. It kept going for another 5 minutes (despite the promise) and then beeped rapidly. 30 seconds later, power went off.

So all in all, it ran for 25 minutes before it died out. The 23 minutes estimate was quite accurate.

Conclusion: Don’t bother calibrating. Just yank the power cord, and measure time.

The third important takeaway is that those “Bull Power” batteries are more like bull-something-else, and I should schedule another battery replacement in a year or so.

And here comes the really funny part. After letting the UPS recharge fully, I checked it again:

$ apcaccess
APC      : 001,027,0652
DATE     : 2021-11-14 15:19:02 +0200
HOSTNAME : thehost
VERSION  : 3.14.14 (31 May 2016) debian
UPSNAME  : theups
CABLE    : USB Cable
DRIVER   : USB UPS Driver
UPSMODE  : Stand Alone
STARTTIME: 2021-11-14 13:50:25 +0200
MODEL    : Smart-UPS 750
STATUS   : ONLINE
BCHARGE  : 100.0 Percent
TIMELEFT : 34.0 Minutes
MBATTCHG : 5 Percent
MINTIMEL : 3 Minutes
MAXTIME  : 300 Seconds
ALARMDEL : 30 Seconds
BATTV    : 26.8 Volts
NUMXFERS : 0
TONBATT  : 0 Seconds
CUMONBATT: 0 Seconds
XOFFBATT : N/A
STATFLAG : 0x05000008
MANDATE  : 2018-05-22
SERIALNO : AS1821351109
NOMBATTV : 24.0 Volts
FIRMWARE : UPS 09.3 / ID=18
END APC  : 2021-11-14 15:19:02 +0200

Is this a smart UPS or what? It just had a calibration cycle which ended with losing power after 25 minutes. How could it estimate 34 minutes now? Maybe if I try again I’ll get 34 minutes? And if this is because the PC consumes less power when Linux is running, why was it only 23 minutes before?

Don’t know. I’ve played with this enough.

Second replacement, 16.4.23

I wasn’t happy with those “bulls” batteries, so I went for an early replacement of these. It turns out that it’s quite difficult to find batteries that are anything but a complete no-name. I barely able to get my hands on a Ritar. And when I say barely, I mean that I managed to get batteries from Afik, which is an Israeli electronics shop (אפיק מ.ל.ה בע”מ in Hebrew). This shop pretends to have its own brand, with a battery labeled AK12-7. But I got a photo of a shipping bill, which indicated that the batteries are manufactured by Ritar. So I went for a pair of those. On the invoice I got from the shop, it says RT1270 (this is the datasheet).

According to the datasheet, the discharge time at 0.55C (see discussion about discharge rates in my separate post) should be one hour. My test ran at 105W, which is grossly 105W/24V =~ 4.4A, which is about 0.63C. So I should have expected 50 minutes or so.

These batteries have small (F1) contact terminals too. I didn’t even think about checking that before I ordered them, actually. But the datasheet says that the battery comes with both F1 and F2 terminals.

I’ll skip to the bottom line right away: These batteries were junk, much worse than the previous ones. It’s plain fraud to call these batteries RT1270. Or maybe it has to do with the timestamp on the battery, which says “201022″. What does it mean? Judging from another battery from the same manufacturer (but different model), which had the timestamp “221215″, the new battery in the UPS was manufactured in October 2020. That could explain things.

So to the story: First I tested the old batteries. I put the computer on the GRUB menu and yanked the power supply from the UPS. Actually, I aborted the first experiment and let the batteries refill on the first attempt. The reason was that the (Western Digital) hard disks made the noise of (proper) activity, so I guess they were doing some kind of self-maintenance or something. I suppose they should be able to take a power loss in the middle of whatever they were doing, but I didn’t want to push my luck. So I just waited long enough, and then they went silent again.

The UPS went down after 9 minutes. Its estimation prior to the test was 12 minutes. The estimation was not bad.

I replaced the batteries with the Afik (Ritar) pair (a.k.a. junk), updated the time of last battery replacement on the UPS, and turned it on. Something was apparently wrong. The UPS was connected to proper power supply, and yet the computer didn’t switch on properly: It powered on, and after a few seconds it had some kind of reboot. It wasn’t clear what the problem was. It could also be because of a problematic USB hub, which tends to irritate the computer (partly because it has its own power supply).

At a later stage I discovered that the BIOS had been reset. Maybe because the computer had been powered off for a relatively long time. How did I notice? Because the CPU fans made more noise than usual, and the CPU’s temperature was lower than usual. I’ve changed the settings in the BIOS to silence the fans somewhat (within safe limits, of course). Plus the RGB sequence of the LEDs inside the computer stopped working. Should I change the BIOS’ battery too?

What I did next was a bit random, but at some point the UPS requested me to confirm that the battery had been replaced.

I took out the battery and pushed it in again. Maybe a loose connection? Who knows. Eventually, I connected the UPS to power, turned on the computer, and all ran fine.

The UPS said that the battery’s charge was 100% and promised all kinds of battery times (mostly around 50 minutes).

I then ran a battery test again. It lasted 5 minutes. When the UPS went on again, it gave the battery 28% and promised 14 minutes. Not so smart UPS.

So I let the battery load to 100% again (that took about 90 minutes), and ran a second battery test.

Bottom line: The UPS promised 49 minutes, in reality it held 11 minutes. This promise made sense, as it was probably based upon 0.63C. No chance in the world that this is an RT1270. Not a decently fresh one, anyhow.

So the old batteries, which I considered new held held 23 minutes when they were new, and 9 minutes after roughly a year and a half. These new batteries held 11 minutes as new!

Restarting the UPS, it reported 0% battery (make sense, doesn’t it?) and no 0 minutes runtime. After getting fully charged, the UPS promised 17 minutes. But that was with Linux running, so the power consumption is lower than the test. So it’s a reasonable estimation for a horrible battery.

Conclusion: I need to replace the batteries again, within a year. The computer shuts itself down after 5 minutes, so I guess it will be fine, but this is ridicolous.

Strategy for the next time

After two fiascoes, I’m changing strategy. So these are things I’m going to insist on:

• The marking on the battery should belong to a company that has a more or less proper website, from which I can download a datasheet.
• The datasheet should include discharging characteristics, either in the form of curves or tables, preferably both.
• The battery’s model code, as it appears in the datasheet, should be printed on the battery.
• If the battery is marked with the name of an Israeli importer, it’s by far not as good, even if the importer publishes a datasheet that is possibly identical to the one of the manufacturer.
• The manufacturing date of the battery should be marked on the battery. It should be easy to figure out what it means, and the battery should be fresh of course.
• Prefer an F2 terminal
• Reputable brand is a bonus.

The rationale: A datasheet with discharging characteristics is a commitment. If a large company buys 1000 batteries, and tests a few samples just to find out that they don’t meet the specs, a hefty lawsuit may follow. No sane battery manufacturer will take that risk if it can’t ensure that their batteries perform well.

I couldn’t find such a datasheet for “Bulls Power”, and surely not for that Afik piece of junk. The latter was probably manufactured by Ritar, but not with their commitment.

Not going to insist on: A known brand. Ritar batteries are unavailable in Israel at the moment, an Yuasa’s batteries cost a fortune. Their datasheet shows discharging graphs, of course. But no tables.

But more than anything, it seems like it’s a matter of age. I don’t think anyone manufactures really bad batteries to begin with. I speculate that those junk brands buy old batteries from good manufacturers, and market them with a different name, at a low price.

Who to buy from

Obviously, web shops are the easiest options. If they cooperate with sending me photos of the battery (see below). If not, go for bricks and mortar electricity shops (Erco, for example). Also consider shops and repair shops for motored toy vehicles for small kids. The search word in Hebrew for that is ממונעים (which means “motored”) but is somehow a collective term for these toys.

Is this a Covid-19 thing?

I called up Lion Electronic‘s sales (a company I consider reputable) for a couple of Yuasa batteries (200 NIS each, April 2023), and I was told that they won’t sell me two batteries at that price, because they are imported specially, and that has an extra cost.

My next hope was to get an Aokly 6FM7/6FM9, which are quite available in Israel. However, as of April 2023, two different suppliers told me that it had no manufacturing date printed on it. Which is really weird. So this manufacturer is irrelevant.

I made other significant attempts to get a battery that meets the above criteria in May 2023, and failed: Every time I asked for a photo of the battery and its timestamp, I got no response, or a rather negative one. I got online purchase orders canceled twice while attempting to buy a CSB battery from AlPesek, which is an importer of this battery. After talking with someone at the company about manufacturing date, and putting that request in the comments of the order, the company canceled the order, claiming that they don’t work with end customers (so why was it listed on the website?) and the batteries were delisted from the website. Trying to get the same battery from a web shop (that is, from the same source, but indirectly), I got the order canceled again, with the supposed reason that the minimum purchase for resellers is 10 units.

And there were several other attempts that ended with the other side simply not cooperating with sending me a couple of photos.

At one of Erco‘s shops, I found a VisionNet battery, which is apparently a local brand made up by Telran (a respectful company). The interesting thing about this battery is that it said “Production date 06/2022″ on the battery itself, and the guy at the shop told me that it had arrived a week earlier (and I believe him, the boxes in the storage were new). Also, there are several photos of the battery with this model, but they look different, and none have that timestamp. So it appears like Telran shops around for batteries and put their own marking on them. And that leaves the question how a battery that arrived a week ago is a year old. So the production date is probably something that the real manufacturer promised. Can that be trusted? I don’t know. If there was any battery I thought seriously about trying, it’s a pair of these.

Given this rather odd picture, I speculate that there was a lot of batteries that got stuck in storage, somehow related to Covid-19. And now the manufacturers need to get rid of a whole lot of old, and hence damaged batteries. That’s the best explanation I have to all these batteries with no date marking, plus a lot of weird behavior.

My decision was hence to stop my attempts to replaced the batteries in my UPS for six months, and try again somewhere towards the end of 2023.

Third battery replacement, 17.12.23

I went into Erco‘s shop in Kiryat Ata, and was offered a couple of Vega Power NP12-7 batteries. Never heard about that brand? Neither have I. Neither did this company have a reassuring Internet footprint. But the batteries had an an engraved timestamp saying 10/08/2023 and they had F2 terminals. So I bought a couple of them, at a price of 120 NIS, VAT included. Cheap, in other words.

And the answer to my previous question: Yes, the lack of fresh batteries was most likely a Covid-19 thing.

I ran a detailed discharging test on these two batteries (as well as on the Afik batteries), which I write about in that separate post. Spoiler: The discharging test doesn’t say too much about how the batteries will behave inside the UPS.

First, I wanted to check the situation of the Afik batteries that were already inside the UPS. I brought down the computer to where GRUB waits for prompt, so the UPS reported power consumption was 105W / 150VA. At this point, the UPS promised 12 minutes.

So I yanked the power cord. The UPS began panic beeping after 7:30 minutes, stating 3 minutes were left, which dropped down to 0% and 0 minutes quite soon. Contrary to its pessimistic estimation, the UPS kept up the power for quite long after that, so the total time with power ended up at 22:00 minutes.

Say what? The same batteries held 11 minutes when I first put them in the UPS, and now the time doubled?!

If I don’t want to develop mysterious theories about what happens inside the battery, the explanation could be that the battery was far from being fully charged when I tested it a few months earlier: The UPS claimed that the battery was 100% full after 90 minutes, but it’s quite possible that this estimation was anything but accurate.

It’s also worth citing Power Sonic’s Technical manual: “By cycling the battery a few times or float charging it for a month or two, the highest level of capacity development is achieved. Power-Sonic batteries are fully charged before leaving the factory, but full capacity is realized only after the battery has been cycled a few times or been on float charge for some time”. This is written about another lead-acid battery of course, but maybe it’s a thing that a few months inside the UPS does the battery good.

I replaced the Afik batteries with Vega Power’s. These batteries had been separately charged with another UPS for 48 hours each. After updating the battery replacement date on the UPS (i.e. telling the UPS that there are new batteries inside), the UPS initially promised an uptime of above an hour, and then swayed along until it gradually stabilized at 49 minutes.

Yanked the power cord. All was good and calm until panic beeping started after 28:15 minutes, after which the power went out just a few seconds later. Boom.

The computer was stable on the GRUB menu throughout all this.

So what’s the verdict? I don’t know. The discharging tests that I ran on both pairs of batteries made things even more confusing: Vega Power performed really well, and Afik didn’t worse, but not much worse.

The only clear conclusion is that a UPS discharge test is required every now and then. There’s no way around this.

Repeated discharging test, 22.3.24

Three months later, I ran the same discharging test on the Vega Power batteries. The panic beep began after 22:10 minutes. At 31:30 minutes, the panic beep went off, and then it went on and off sporadically as the UPS’ estimation for remaining time fluctuated between 3 and 6 minutes. At 33:50 minutes the panic beep became steadily on again, and at 34:20 the UPS went off.

In short: Three months of floating current added 6 minutes to the battery time. And an even more confused ride as the battery discharged.