Looking for a UPS Design That Doesn't Overheat Batteries

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.
After working with an electrician, I have a theory about why this is happening, and if correct, the theory suggests a different design for UPS systems that would avoid the problem. I am hoping some manufacturer has already implemented this idea and someone can refer me to their products.
On all of the UPS systems we use generic "brick" batteries are joined together in a series, then the leads from the ends of these battery chains are connected to the UPS. The problem is that batteries rarely fail all together. If a 12V battery should be considered discharged and not useful at around 10V, and you have two 12V batteries joined in series, what happens when one of the batteries maintains a full charge at 12V but the other battery in the series starts to lose its ability to hold charge and slips below some critical level? From the point of view of the UPS, it doesn't see anything about the state of individual batteries. The UPS only sees that the total voltage of the two 12V batteries in series has fallen from 24V to 22V.
Maybe an electrical engineer can step in here and explain what is happening, but my pure guess is that to maintain the same power output, an increased amount of current probably has to flow through the batteries. That creates problems with heating for the "good" battery, which is still measuring 12V. Now that 12V is receiving too much current, overcharges, overheats, and at some point the casing of the battery starts to melt. I haven't done enough experimentation to determine if it is the good battery or bad battery that is overheating. To be honest, in such situations I have often seen evidence that both batteries start to melt. Perhaps this is nothing more than one battery being in physical proximity to the overheating battery and therefore gaining heat from its physical contact. The only thing that is common to all cases is that one of the two batteries has discharged and should have been replaced before the overheating event took place.
Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.
Aside from increasing safety and utility of the monitoring system, such a design would allow much easier re-use of off-the-shelf batteries, improving ease-of-use in making battery changes and lowering cost. While I realize that APC in particular has no desire to make anything regarding batteries non proprietary, maybe some other vendor has a UPS design that puts a direct monitoring circuit on each individual 12V brick battery, thus avoiding the overheating problem I have described?
Any information on why this overheating takes place, how to avoid it, and any referrals to third party UPS products that employ a more robust design are appreciated.
-- Will

Seen this before, most UPS dont have individual temp sensors on each battery and/or dont have them in the right place and/or maybe more than one per battery is needed. I have a solution I have been working on with a battery manufacturer which I cant discuss for commercial reasons but I can say we havent pursued it much because the issue hasnt been so major - its just been 'occasional' and didnt warrant a whole new topology, this might have changed, so thanks for your report. There are intermediate fixes you can do for any UPS if you can open it up, so I'd suggest that route first, esp for UPS that are mostly unattended. I heard of one situation where an office was left overnight with the UPS functioning online and the owner entered the premises in the morning to find the room was quite warm and a bad acrid smell throughout. Turned out the batteries had been overcharged because one or more cells had shorted and the UPS wasnt the smart one they said it was <sigh>
Cheers
-- Regards Mike * GMC/VL Commodore, Calais VL Turbo FuseRail that wont warp or melt ! * High grade milspec ignition driver electronics now in development * Twin Tyres to suit most sedans, trikes and motorcycle sidecars http://niche.iinet.net.au

In article , westes-usc@noemail.nospam says...

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.
After working with an electrician, I have a theory about why this is happening, and if correct, the theory suggests a different design for UPS systems that would avoid the problem. I am hoping some manufacturer has already implemented this idea and someone can refer me to their products.
On all of the UPS systems we use generic "brick" batteries are joined together in a series, then the leads from the ends of these battery chains are connected to the UPS. The problem is that batteries rarely fail all together. If a 12V battery should be considered discharged and not useful at around 10V, and you have two 12V batteries joined in series, what happens when one of the batteries maintains a full charge at 12V but the other battery in the series starts to lose its ability to hold charge and slips below some critical level? From the point of view of the UPS, it doesn't see anything about the state of individual batteries. The UPS only sees that the total voltage of the two 12V batteries in series has fallen from 24V to 22V.
Maybe an electrical engineer can step in here and explain what is happening, but my pure guess is that to maintain the same power output, an increased amount of current probably has to flow through the batteries. That creates problems with heating for the "good" battery, which is still measuring 12V. Now that 12V is receiving too much current, overcharges, overheats, and at some point the casing of the battery starts to melt. I haven't done enough experimentation to determine if it is the good battery or bad battery that is overheating. To be honest, in such situations I have often seen evidence that both batteries start to melt. Perhaps this is nothing more than one battery being in physical proximity to the overheating battery and therefore gaining heat from its physical contact. The only thing that is common to all cases is that one of the two batteries has discharged and should have been replaced before the overheating event took place.
Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.
Aside from increasing safety and utility of the monitoring system, such a design would allow much easier re-use of off-the-shelf batteries, improving ease-of-use in making battery changes and lowering cost. While I realize that APC in particular has no desire to make anything regarding batteries non proprietary, maybe some other vendor has a UPS design that puts a direct monitoring circuit on each individual 12V brick battery, thus avoiding the overheating problem I have described?
Any information on why this overheating takes place, how to avoid it, and any referrals to third party UPS products that employ a more robust design are appreciated.
-- Will

Will wrote:

Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.

Or multiple emails, as in:
while true do mail westes-usc@noemail.nospam < replacemybatteryfile sleep 3600 done
An overtemp shutdown might be simpler :-)
Nick

Will wrote:

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.
After working with an electrician, I have a theory about why this is happening, and if correct, the theory suggests a different design for UPS systems that would avoid the problem. I am hoping some manufacturer has already implemented this idea and someone can refer me to their products.
On all of the UPS systems we use generic "brick" batteries are joined together in a series, then the leads from the ends of these battery chains are connected to the UPS. The problem is that batteries rarely fail all together. If a 12V battery should be considered discharged and not useful at around 10V, and you have two 12V batteries joined in series, what happens when one of the batteries maintains a full charge at 12V but the other battery in the series starts to lose its ability to hold charge and slips below some critical level? From the point of view of the UPS, it doesn't see anything about the state of individual batteries. The UPS only sees that the total voltage of the two 12V batteries in series has fallen from 24V to 22V.
Maybe an electrical engineer can step in here and explain what is happening, but my pure guess is that to maintain the same power output, an increased amount of current probably has to flow through the batteries. That creates problems with heating for the "good" battery, which is still measuring 12V. Now that 12V is receiving too much current, overcharges, overheats, and at some point the casing of the battery starts to melt. I haven't done enough experimentation to determine if it is the good battery or bad battery that is overheating. To be honest, in such situations I have often seen evidence that both batteries start to melt. Perhaps this is nothing more than one battery being in physical proximity to the overheating battery and therefore gaining heat from its physical contact. The only thing that is common to all cases is that one of the two batteries has discharged and should have been replaced before the overheating event took place.
Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.
Aside from increasing safety and utility of the monitoring system, such a design would allow much easier re-use of off-the-shelf batteries, improving ease-of-use in making battery changes and lowering cost. While I realize that APC in particular has no desire to make anything regarding batteries non proprietary, maybe some other vendor has a UPS design that puts a direct monitoring circuit on each individual 12V brick battery, thus avoiding the overheating problem I have described?
Any information on why this overheating takes place, how to avoid it, and any referrals to third party UPS products that employ a more robust design are appreciated.

If yoru UPS is not detecting this then get a different UPS. Just because you supposedly have a good brand does not mean you actually have a good brand. Especially when you single handedly identify a flaw in their system.
In fail open situation the low battery will not absorb the energy and the voltage will raise right away. You will find our your batt is bad when you try to use it and it runs out quickly.
In fail short situation the low battery will absorb the energy and the voltage will not raise and the batts will slowly heat up. It should be detected.
Looks like they just are not checking for fail short.
How can there be enough heat to melt a battery but the temp sensor system not complain?

On Wed, 31 May 2006 21:45:02 -0700, "Will" wrote:

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.
After working with an electrician, I have a theory about why this is happening, and if correct, the theory suggests a different design for UPS systems that would avoid the problem. I am hoping some manufacturer has already implemented this idea and someone can refer me to their products.

IMOE the main reason for the failure of SLA/VRLA batteries in SOHO UPS units is the charging regime - no more, no less.
Recovery after a discharge is a tradeoff. Do you want minimum recovery to (substantially) full charge to best position you for a possible follow-up outage, or a leisurely recovery which treats the battery properly?
The bad news is that you don't get to choose - the UPS manufacturers have made that decision for you and have opted for the over-zealous rate. This is usually compounded by an excessively high float voltage. To further shorten the life of the batteries, the regime is often one of current-limited (too high) constant voltage (also too high) charging. An example of a far better regime is set out in the data sheet for the Unitrode/TI UC2906/3906 SLA charge controller chips. Unfortunately - and yes, there seems to be only bad news - it is usually impossible to splice in a better charging system without causing the UPS' monitoring circuits to go apeshit.

On all of the UPS systems we use generic "brick" batteries are joined together in a series, then the leads from the ends of these battery chains are connected to the UPS. The problem is that batteries rarely fail all together. If a 12V battery should be considered discharged and not useful at around 10V, and you have two 12V batteries joined in series, what happens when one of the batteries maintains a full charge at 12V but the other battery in the series starts to lose its ability to hold charge and slips below some critical level? From the point of view of the UPS, it doesn't see anything about the state of individual batteries. The UPS only sees that the total voltage of the two 12V batteries in series has fallen from 24V to 22V.

Indeed it would be good sense to monitor the individual batteries, but these are price-sensitive products in the SOHO sizes. Certainly there is scope to add a monitor which reports on voltage differential.

Maybe an electrical engineer can step in here and explain what is happening, but my pure guess is that to maintain the same power output, an increased amount of current probably has to flow through the batteries. That creates problems with heating for the "good" battery, which is still measuring 12V. Now that 12V is receiving too much current, overcharges, overheats, and at some point the casing of the battery starts to melt.

You are mixing up discharge and charge in the above, UIAVMM.

I haven't done enough experimentation to determine if it is the good battery or bad battery that is overheating. To be honest, in such situations I have often seen evidence that both batteries start to melt. Perhaps this is nothing more than one battery being in physical proximity to the overheating battery and therefore gaining heat from its physical contact. The only thing that is common to all cases is that one of the two batteries has discharged and should have been replaced before the overheating event took place.

By the time one battery in a series string needs replacing, they all should be replaced. You create the exact situation you referred to above by replacing one of a series string.

Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.
Aside from increasing safety and utility of the monitoring system, such a design would allow much easier re-use of off-the-shelf batteries, improving ease-of-use in making battery changes and lowering cost. While I realize that APC in particular has no desire to make anything regarding batteries non proprietary, maybe some other vendor has a UPS design that puts a direct monitoring circuit on each individual 12V brick battery, thus avoiding the overheating problem I have described?
Any information on why this overheating takes place, how to avoid it, and any referrals to third party UPS products that employ a more robust design are appreciated.

The other consideration that compounds battery heating is the way they are tightly packed into the case. Again, short of mounting them outside - as some users do - there is little that can be done in mitigation as there simply isn't room to provide ventilation or additional conduction paths.
Both of the UPS units I have here - 1000VA (5x12V) and 300VA (2x12V) - have the batteries almost shoe-horned into their place. In fact, there is no way a shoehorn could fit in there between them. What I have done, though, is re-engineer the charging circuitry to "better" recovery and float charging conditions.

"Will" wrote:

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking.

Are you replacing the batteries every few years as per the manufacturer's reccomendations? New batteries are much cheaper than the elaborate battery monitoring, charging, and warning system you want to engineer.
If this is a Giant UPS for (say) a machine room, there are are other options, but for small individual UPSen, you aren't going to find what you want.

"Will" wrote in message

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.

Best I can offer, is to avoid deep discharges that can sulfate and short the plates. Then avoid putting batteries in parallel, that can short through those sulfated plates. Maybe a fusible link between the banks would burn up instead of the batteries themselves. Another idea is to put 8-10 cells in series, to give you 120V (number varies with type of cells used) and then use the inverter alone, without the transformer to convert it to AC.

Will wrote:

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat ...

There's not much to add to the good comments already given. The important concept is *battery*, i.e., a group of cells. Each 12-volt battery is 6 2-volt cells in series. It makes no difference if the 20 cells of a 120-volt battery are in one case or several provided they are well matched. Mixing cells from different production lots is not good form unless production tolerances are very tight. The only safe way to use a mix of cells of varying capacity is by charging the cells individually and discharging them in parallel. Then you need to make sure that the discharge is stopped when any one cell is depleted.
That's just not practical. You wouldn't mix old and new cells in a flashlight. Don't do it in a UPS.
Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������

"Will" wrote :

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have been lucky to catch such thermal events with temperature sensors but it has always been a goal of mine to better understand why this happens, and to find some UPS system where it can be avoided entirely. To date, we have seen these problems with APC Symmetra tower, Symmetra rackmount, and SmartUPS.
After working with an electrician, I have a theory about why this is happening, and if correct, the theory suggests a different design for UPS systems that would avoid the problem. I am hoping some manufacturer has already implemented this idea and someone can refer me to their products.
On all of the UPS systems we use generic "brick" batteries are joined together in a series, then the leads from the ends of these battery chains are connected to the UPS. The problem is that batteries rarely fail all together. If a 12V battery should be considered discharged and not useful at around 10V, and you have two 12V batteries joined in series, what happens when one of the batteries maintains a full charge at 12V but the other battery in the series starts to lose its ability to hold charge and slips below some critical level? From the point of view of the UPS, it doesn't see anything about the state of individual batteries. The UPS only sees that the total voltage of the two 12V batteries in series has fallen from 24V to 22V.
Maybe an electrical engineer can step in here and explain what is happening, but my pure guess is that to maintain the same power output, an increased amount of current probably has to flow through the batteries. That creates problems with heating for the "good" battery, which is still measuring 12V. Now that 12V is receiving too much current, overcharges, overheats, and at some point the casing of the battery starts to melt. I haven't done enough experimentation to determine if it is the good battery or bad battery that is overheating. To be honest, in such situations I have often seen evidence that both batteries start to melt. Perhaps this is nothing more than one battery being in physical proximity to the overheating battery and therefore gaining heat from its physical contact. The only thing that is common to all cases is that one of the two batteries has discharged and should have been replaced before the overheating event took place.
Regardless of the actual mechanism for the overheating we are observing, it seems to me that the obvious solution is to design UPS systems to physically connect to each 12V battery individually. Forget connecting multiple batteries in series, at least don't do that at the battery itself. By connecting to and monitoring individual batteries, now the UPS can see when an individual battery falls below some critical voltage threshold and put it into a special recharge state (not put any load on it). If the battery fails to recharge, the UPS can declare the battery defective and can signal the condition by an LED on the battery's compartment. If there is a network attached monitoring system, the UPS can send an e-mail.
Aside from increasing safety and utility of the monitoring system, such a design would allow much easier re-use of off-the-shelf batteries, improving ease-of-use in making battery changes and lowering cost. While I realize that APC in particular has no desire to make anything regarding batteries non proprietary, maybe some other vendor has a UPS design that puts a direct monitoring circuit on each individual 12V brick battery, thus avoiding the overheating problem I have described?
Any information on why this overheating takes place, how to avoid it, and any referrals to third party UPS products that employ a more robust design are appreciated.
-- Will

Here is an example of one thing to consider. In a string of cells (or batteries) you could have one that over time has fallen in usable capacity faster than the others. In the situation where the entire string is being discharged, the moment that the weakest cell runs out of capacity it will no longer be contributing to the load and will essentially be CHARGED BACKWARDS by the discharge current of the remaining cells.
You will get plenty of gassing and other bad things happening under this scenario. Depending on the design of the system, it may not even be able to detect that a cell has effectively dropped out. Obviously this is a bigger problem in higher voltage battery configurations. For example, you will notice a dead cell in a 4 cell pack much easier than in a 12 cell pack.
(*>

"Will" wrote in message

Our company has had a long-standing problem where UPS batteries will at various points in their lifetime suddenly overheat, sometimes catastrophically to the point where the battery casing starts to melt and you can actually smell the gases from the battery leaking. So far we have

Thank you to everyone for all the good comments made so far in this thread. The bottom line seems to be that most smaller UPS systems use a too-aggressive charging scheme, and pack the batteries too close together, so that thermal events in one battery quickly spread to other batteries touching that one.
Does any UPS manufacturer make a unit that will do about 3000VA - rackmount or tower configuration - that will treat the batteries better and use less aggressive charging in favor of long battery life? And of those units, does anyone make a unit where the batteries are given enough space between them that overheating in one battery stays localized to that battery?
-- Will

On Thu, 1 Jun 2006 11:59:22 -0700, "Will" wrote:

Does any UPS manufacturer make a unit that will do about 3000VA - rackmount or tower configuration - that will treat the batteries better and use less aggressive charging in favor of long battery life? And of those units, does anyone make a unit where the batteries are given enough space between them that overheating in one battery stays localized to that battery?
-- Will
I don't know what is available, but if this is what you want (3000VA),

it seems to me that you might be better to put together your own UPS from discrete units. It might be larger (do you have room for it?), but it ought to be more reliable. Find a reasonable inverter, perhaps one that runs on 12volts (to mitigate needing too many "cells"), and perhaps use about 20 to 30 amphr SLA (or even bigger) with a quality seperate charger - something that can provide the power you normally need for you computers, but also has a proper float charging regime. Run everything through the inverter all the time. (Personally I prefer "wet cells" like T105's - but that's probably not suitable in an office situation - it just that I think they are harder to kill). It will be more expensive, but it much easier to monitor. You might even have a staff member learn to measure the the battery voltage on a regular basis.
My tuppence worth
Eric Sears

One idea that has been missed so far: If you are really serious about reliability (most UPS users aren't) and don't like dealing with batteries, consider installing a Rotary UPS. You'll find plenty of good info via a Google search.
I know it's "old school" and more expensive initial outlay, but Rotary UPS's are used by the majority of the world's Stock Exchanges and major data centres for all of the reasons you mentioned in your post - and on power failure, they just work.
HTH, Cameron:-)

"Cameron Dorrough" wrote:

One idea that has been missed so far: If you are really serious about reliability (most UPS users aren't) and don't like dealing with batteries, consider installing a Rotary UPS. You'll find plenty of good info via a Google search.

Don't Rotary UPSes just provide a few seconds of power while your diesel generators are firing up?

"William P.N. Smith" wrote in message

"Cameron Dorrough" wrote: One idea that has been missed so far: If you are really serious about reliability (most UPS users aren't) and don't like dealing with batteries, consider installing a Rotary UPS. You'll find plenty of good info via a Google search.
Don't Rotary UPSes just provide a few seconds of power while your diesel generators are firing up?

I'm sure some are configured that way - that would be a TPS (Temporary Power Supply). Although it takes a lot longer than a few seconds to run the flywheel down.
It depends what you buy. The Pillar UPS powering the Melbourne Stock Exchange (that's Melbourne, Australia - not Melbourne, USA) is rated for 30 minutes at full load.
Cameron:-)

On Thu, 01 Jun 2006 19:58:26 GMT, phoneme@025379386.for.email.address (Eric Sears) wrote:

I don't know what is available, but if this is what you want (3000VA), it seems to me that you might be better to put together your own UPS from discrete units. It might be larger (do you have room for it?), but it ought to be more reliable. Find a reasonable inverter, perhaps one that runs on 12volts (to mitigate needing too many "cells"), and perhaps use about 20 to 30 amphr SLA (or even bigger)

(snip)
???? a 3000VA inverter is going to want 250A @12V input, assuming 100% efficiency. There IS a reason why higher input voltages are used as the output rating climbs. Even my 1000VA unit runs a 60V string of 5*12V.