Data centers have become one of the most critical developments in the technology world. The explosion of communication, Smartphone applications, social media, cloud computing and the Internet of Things (IoT), allows individuals to weave the Internet into their daily routines. Also, businesses store and mine big data to become more productive in solving challenges long thought to be unsolvable.
The Internet allows everyone to work virtually, connect with each other globally, to exchange information and data in real time, and enables everyone to harvest and analyze large data to become much more productive. This, in turn, requires data managers to store more data in their data centers. So much so that the number of data centers has doubled approximately every five years and will continue to grow exponentially. Add in the growing IoT need for devices to always be connected and exchanging data at high volumes, managing data centers effectively is, and will always be, mission critical.
Design engineers, operators and owners face many challenges when designing, building and managing today’s data centers. The electrical distribution system is a key consideration when designing within the whitespace. To ensure maximum uptime, a very complex electrical distribution system is required to support the data center from incoming power down to the server racks, with each system portion presenting its own challenges.
Total cost of ownership is one of the first challenges that the designer is confronted with. It’s estimated that the electricity needed to power the entire system represents 50% of the annual operating cost. With electrical power consumption this large, there are many ways to measure how efficiently it’s being used. Power Usage Effectiveness (PUE) is a popular metric to quantify this. PUE is a scale for how efficiently a data center uses power for its IT equipment versus the total power consumed, expressed as:
PUE = Total power consumption
IT equipment power consumption
The goal is to get the PUE value as close as possible to 1.0. In recent studies and surveys with data center owners and operators, it’s been noted the average PUE is around 2.0, meaning that roughly half the data center’s power consumption isn’t powering the IT equipment. To lower this ratio, many designers tend to focus on the energy used for cooling and auxiliary power as that’s non-value-added power consumption. However, one must not lose sight of using the “effective” power more efficiently as it can indirectly affect the amount of non-value-added power consumed within the whitespace and result in a more positive affect on total power consumed. Increasing the system voltage is a way to achieve this. By removing transformers that step power down to 208 V, or lower, can result in 2% or more efficiency gains. While today’s servers are smaller, they still require the same amount of power. With more servers installed per rack, each rack is now consuming more total power. In turn, higher system voltages provide efficiency gains down to the rack level that can result in reduced space for the same computing capacity, or greater computing capacity in the same space. The table below shows the power delivered to the server rack based on the volts and amps supplied from the busway or Remote Power Panel (RPP) to the rack Power Distribution Unit (PDU).
The table above shows that higher voltages can greatly improve the power density at the rack level. By integrating this concept into the designs for medium to large data centers, designers and engineers can lower the up-front capital costs for the electrical distribution system. This also includes reducing the overall whitespace square footage, the number of installed transformers and conductor sizes, etc. All the advantages of higher voltages can decrease the total ownership cost by reducing the initial capital to build a data center and the yearly operating expenses.
There can be a caveat by moving to electrical systems with higher density and efficient power distribution, and that is the available fault current experienced throughout the whitespace. Since more power is delivered with lower impedance, fault currents can be much higher than in lower voltage designs. Per the NEC and to ensure safety, designers must calculate available fault current levels at multiple points in the system. Circuit protection must then be incorporated into many points in the system to protect its components and, more importantly, the people servicing it. Determining adequate circuit protection is an issue faced by many designers when creating higher power density systems.
Another challenge faced by data center engineers is to ensure their electrical system design is flexible and scalable. During a data center’s life cycle, IT refreshes are typically required to keep up with increasing demands. Also, multi-tenant data centers are rapidly increasing with an expected 15% compound annual growth rate that outpaces the overall data center growth rate. As companies lease server space within a data center, the data center owner must accommodate changing power needs all the way down to the rack level. Ensuring that power distribution is scalable throughout the system requires the accompanying circuit protection is also scalable.
For fixed, non-removable busway drops and RPPs with circuit breakers, having a scalable solution requires the owner to shut down the upstream overcurrent protective device (OCPD) in order to service or replace downstream devices, or replace the busway drop altogether. This causes downtime that’s more widespread than necessary (which can be costly) or require extensive planning to execute. The issue faced by designers is how to future-proof their electrical system’s design so that it’s scalable to meet the inevitable increasing power demands. Products that allow designers to accommodate such changes are increasingly in demand.
Fusible design benefits
Modern fuse solutions continue to evolve with better designs, materials, physical size and service features to provide critical protection for the demanding requirements of variable speed drives, UPSs, photovoltaic systems, electrical vehicle power and control systems, building emergency systems and critical operations power systems. They also have many features that benefit designers, maintainers and owners of today’s and tomorrow’s data centers.
These advantages are more pronounced when considering the challenges posed by data center trends to higher energy efficiency and greater power density. Consequently, OCPDs in the PDUs, RPPs, busway plug-in units and rack PDUs must have higher interrupting ratings. The electrical equipment, such as PDUs/RPPs and busway plug-in units, must typically have higher short-circuit current ratings. Additionally, for data centers with higher fault currents, ensuring selective coordination becomes more critical for avoiding unnecessary system outages. Similarly, higher voltages and fault currents generally demand higher performance OCPDs and greater attention paid to electrical safe work practices. For instance, when using non-current limiting OCPDs, higher fault currents result in higher arc flash incident energy, posing a greater safety risk. However, this is not necessarily the case with current-limiting fuses that can reliably mitigate arc flash incident energy, even for high fault currents.
Fuse benefits are divided into four general topics:
- Electrical installation safety requirements
- Safe work practices
Electrical installation safety requirements
There are mandatory NEC and OSHA safety requirements for electrical equipment that must be met upon the initial installation and throughout the system’s life cycle. Most people recognize that electrical equipment voltage ratings must be equal to or greater than the system voltage. However, two other critical ratings include:
- Interrupting Rating (IR)
- Equipment Short-Circuit Current Rating (SCCR)
Interrupting Rating (IR) is the maximum fault current at a specified voltage that a circuit breaker or fuse can safely interrupt under standard test conditions. All OCPDs must have IRs equal to or greater than the available fault current at their lineside terminals per NEC 110.9 and OSHA 1910.303(b) (4). It is a serious safety hazard to have an OCPD installed where the available fault current exceeds the OCPD’s IR. If a circuit breaker or fuse attempts to interrupt a fault current exceeding its IR, the device can catastrophically fail in a violent explosion.
Along with fault current levels trending up in new data centers, existing data centers can be expanded, increasing fault current levels beyond the existing OCPDs’ IRs. Current-limiting fuses typically have IRs of 100 kA, 200 kA or 300 kA for 600 Vac or less. With higher voltage distribution to the cabinet (i.e., 415 Vac or greater), it’s not uncommon to have 20 kA to 50 kA or more available fault current on a RPP or busway plug-in unit. Even the rack PDUs can have high fault currents.
Therefore, the 5 kA IR or 10 kA IR circuit breakers, often used in rack PDUs, may be inadequate for many installations.
Circuit breaker solutions for higher available fault currents are either:
- Fully rated circuit breakers (each circuit breaker has an individual IR equal or greater than the available fault current at its lineside terminals)
- Series combination rated circuit breakers (a circuit breaker is permitted to have an IR less than the available fault current at its lineside terminals if installed in a panelboard that is tested, listed and marked with a specific lineside circuit breaker or fuse)
Fully rated circuit breakers with higher IRs cost more and may have a larger footprint. In either case, with fully rated or series rated circuit breaker systems, achieving selective coordination is usually more difficult than when using standard molded case circuit breakers, and this is more pronounced as the fault currents and voltages increase.
Modern current-limiting fuses inherently provide high, fully rated IRs for systems with fault currents up to 100 kA or 200 kA without any price premium or footprint increase. All high IR current-limiting fuses are simple to selectively coordinate and easily provide excellent circuit component protection. In addition, each current-limiting fuse class has unique physical dimensions so that replacement fuses will always be of the same fuse class and will maintain the safety of high Interrupting Ratings.
Equipment Short-Circuit Current Rating (SCCR)
A principal advantage to fusing data center circuits is the current-limiting fuse’s ability to greatly reduce the let-through energy during faults. Per UL 248 fuse product standard, current-limiting fuses are not permitted to exceed maximum allowable energy let-through values under fault conditions. This provides excellent component protection and is a safety benefit. The most current-limiting Bussmann series fuses provide superior fault current protection. All equipment and components in the data center’s electrical system are required to have an SCCR equal to or greater than the available fault current. This includes transfer switches, UPSs, PDUs, RPPs, busways, bus plug-in units, computer room air conditioner (CRAC) units and row coolers, etc.
Most fusible panelboards, busway systems and enclosed disconnects can be tested, listed and marked with a 200 kA SCCR. E.g., when not protected by current-limiting fuses, a common 250 A data center busway system has a standard SCCR of 22 kA. This same busway, when protected upstream by 400 A or less Class J fuses, has been tested and listed with a 200 kA SCCR at 600 V or less.
As a result of higher efficiencies and higher power density designs electrical systems are capable of delivering higher fault current levels. A key current-limiting device benefit when a fault occurs is to quickly drive the fault current down to zero and keep the let-through energy below levels that damage equipment.
Long before it was an NEC requirement, preventing an unnecessary power outage has been a design consideration in data centers and mission critical facilities. Mission critical system designers understand the added reliability that selectively coordinated OCPDs provide.
Using properly selected and applied fuses in data centers alleviates the design hassle of trying to achieve selectively coordinated OCPDs at the cabinet and busway (or PDU) levels as well as further upstream. To achieve selective coordination, fuses simply need to maintain a 2:1* amp rating ratio for upstream and downstream Bussmann series Low- Peak fuses. This eliminates the possibility of cascading multiple OCPD levels under fault conditions.
When OCPDs are not selectively coordinated, multiple, upstream OCPD levels can cascade open on a fault condition. For example: a fault in a power supply or rack PDUs results not only in one of the rack PDU OCPDs opening (as it should), but the OCPDs in RPP or busway plug-in units unnecessarily open, resulting in the unnecessary power outage to the entire rack PDU. Even worse results would be if the much larger amp rated feeder OCPD (200 to 400 A) supplying the busway or PDU/RPP should open for a fault in the rack PDU, there would be an unnecessary power outage to an entire PDU/RPP or busway run.
Inherent fuse reliability
Fuse operation is based on a simple thermal principle; the fuse element will rapidly melt or vaporize and open at a very specific energy level. Users can be assured the fuse element’s precise thermal attributes will always operate when called upon to remove a fault and protect valuable equipment. The internal parts of modern current-limiting fuses do not require maintenance so that the ongoing maintenance cost of a fusible system is typically less. However, it remains important to periodically inspect electrical equipment for signs of overheating, poor connection or insufficient conductor ampacity.
If an overcurrent causes an overload triggering the fuse to open, then fuses can be replaced, or circuit breakers reset after the cause has been identified and remedied. However, if a faulted circuit caused the opening, then fuses cannot be replaced, nor circuit breakers reset “…until it has been determined that the equipment and circuit can be safely energized.” To avoid possible catastrophic damage to equipment or unsafe working conditions, it is important to identify the source of the fault and repair the faulted circuit before restoring power. Additionally, the conductors and electrical components on the faulted circuit path should be verified suitable to be placed back into service. When a fuse opens an overcurrent, it is replaced with a new, factory-calibrated fuse, ensuring the same protection level, which is an advantage of using fuses.
If a circuit breaker opens a fault approaching its interrupting rating, the circuit breaker must be inspected and tested in accordance with manufacturer’s instructions and determined to be operating within specifications before it can be placed back into service.
Arc flash reduction
Arc flash is a frequent concern in today’s data centers. With minimizing downtime, a priority, it is important wherever possible to have current- limiting OCPDs to mitigate arc flash hazards. By limiting energy let-through and quickly bringing current down to zero, fuses can reduce the arc flash incident energy experienced during most arc flash events. This especially pertains to the fuse types and amp ratings used in rack PDUs (typically 20 A current-limiting fuses) and busway plug-in units (typically 100 A or less current-limiting fuses). This also pertains to the busway when protected by fuses in the end-feeds.
In addition, mitigating arc flash hazard depends on the OCPD’s design and condition of maintenance. If OCPDs requiring maintenance are not maintained, an arc flash event can be more severe than that predicted by an arc flash risk assessment. OCPDs must be maintained and the maintenance, tests, and inspections to be documented. Fuses are inherently reliable for fault conditions as there is no need to maintain the fuses’ internal parts. All that is necessary is to maintain the external connections and proper environmental conditions.
Current-limiting fuse safety system
By design, a given UL class current-limiting fuse cannot be inserted into mountings designed for another fuse class. As a result, fuses with lower voltage and interrupting ratings, (or lower current limitation performance) can’t be accidentally put into service. This is a significant safety benefit when utilizing current- limiting fusible equipment. Rejection features for fuse amp ratings can vary based on the fuse class and the type selected made by the designer and installer. Each approach provides different benefits, depending on the UL fuse classes used in the system design.