White Paper: Power Quality Pain Points

IMPROVING DATA CENTER RELIABILITY AND EFFICIENCY BY SOLVING POWER QUALITY PAIN POINTS


Download PDF

EXECUTIVE SUMMARY

DATA CENTER POWER PAIN POINTS
  1. Access to granular data on power consumption
    and power quality
  2. Knowing when hidden power quality issues
    are leading to other problems
  3. Determining outlet-level conditions when
    working in the back of the rack
  4. Identifying stranded power capacity
  5. Support for a wide variety of DCIM tools
    through Redfish API and SNMP
  6. Having the right outlet type in the
    right place
  7. Insecure power cords that work loose
    due to vibration

Electrical power is the lifeblood of the data center. A stable source of clean power must be present for any data center to operate. But the very equipment that runs in the data center is often the cause of power quality problems experienced throughout that data center. Fans, power supplies, rectifiers, transformers, pumps, and lighting ballasts can distort the available power waveform. The resulting sags, transients, and harmonics may make the infrastructure unreliable and inefficient.

power waveform

As a specialist and innovator in the data center power business, Legrand has taken a giant leap forward in the functionality of our power distribution units by integrating new continuous power quality monitoring capabilities. This paper explains why measuring total harmonic distortion (THD) is essential to modern data centers and how having circuit breaker trip forensics available at the rack PDU makes sense to address several data center power pain points.

INTRODUCTION

The hardware technology employed in modern data centers is typically faster and more power dense than prior generations of infrastructure. CPUs now have tens to hundreds of cores in a single chip, and GPUs are likely to have thousands of cores. Sophisticated software is available to make use of all these cores in parallel, reducing the overall time to perform a given task. When used together, billions or trillions of operations per second may be performed in a single server enclosure running countless virtual machines or containers. Unplanned interruptions due to power problems can set back a massive amount of work and result in downtime and lost revenue for the data center. It is in this environment that we most often see the impacts of unexpected power quality degradation.

Historically, assessing power quality has been undertaken during the commissioning and bring-up phase of the data center as a one-time event utilizing manually placed sensors, and would likely not occur again until a massive server refresh cycle is undertaken. The facilities and IT teams of the data center both must participate, with electricians on standby to troubleshoot and make corrections.

With the ongoing focus on improving both efficiency and sustainability of the data center, more frequent or even continuous power quality monitoring becomes desirable. Continuous monitoring facilitates having a dynamic operating environment rather than a steady state, letting machines be turned off, on, or throttled back to support grid-interactive and follow-the-sun operation models of the data center. Access to near real-time granular power quality information helps the facility and IT managers quickly resolve identified problems, improving uptime and power utilization efficiency.

WHAT IS POWER QUALITY?

TOP TAKEAWAYS
  1. Data center hardware often introduces its own
    power quality problems
  2. The PUE and uptime of your data center is impacted
    by the presence of power deviations
  3. Deviations from an ideal power wave take many forms
    such as sags, transients and harmonics
  4. Unwanted harmonics reduce reliability, decrease uptime
    and result in higher energy costs
  5. Tools are available that enable continuous power quality
    and THD measurements to be taken
  6. Solutions exist for reducing or eliminating the effects
    of harmonics and other power deviations

In general terms, power quality is a means of expressing the stability of a power source and its ability to deliver ideal waveforms that stay within the specified range to equipment in the data center facility, enabling the infrastructure to operate properly. Most engineers and facilities managers think of the quality of the voltage waveform as “power quality”¹ because the current in a circuit is a dependent value, contingent on the supply voltage and the load being powered. The type of loads and their distribution across the data center can impact the quality of power available to the other equipment in the data center.

Power quality is a multi-dimensional measurement that considers the voltage sags, surges/swells, transients, interruptions, total harmonic distortion (THD) and other power problems that result in complications for the equipment in the data center. By comparing actual measurements relative to the ideal, we can express the power quality in a facility, all the way down to the branch or device.


SOURCES OF POWER QUALITY ISSUES

Within the data center are many potential causes of power quality issues. The following, although not comprehensive, is a list of data center equipment that has the latent ability to cause problems to the voltage waveform:

  • Non-linear switch mode power supplies used for servers, switches and storage systems can emit harmonics backward into the power distribution infrastructure, especially older ones connected phase-to-neutral instead of phase-to-phase.
  • Variable frequency drive (VFD) fan motors found in HVAC systems.
  • Pumps used to move liquids found in cooling systems, whether D2C, RDHx , or others.
  • Lighting ballasts (used with older fluorescent lighting fixtures).
  • Rectifiers, whether on the input to UPS systems, or used as a common “power shelf” feeding DC power to a busbar in the rack.
  • UPS inverters that turn DC power back into AC.
  • DC-to-DC converters that exhibit AC wave forms superimposed on the DC output (known as AC ripple).
  • Generators that do not have an impedance that is identical to that of the utility power source.

EFFECTS OF HARMONICS

voltage waveform of a data center

The adjacent figure from the Green Grid’s paper “Data Center Power System Harmonics”² illustrates both an ideal voltage waveform and a waveform with harmonics present.

Let’s look at what may happen when unacceptable levels of harmonics are present in the voltage waveform of the data center:

  • Harmonics can negatively impact equipment reliability, uptime and energy costs.3
  • Resultant harmonic currents can cause false circuit breaker tripping due to thermal or electronic trips.4
  • Harmonic voltages can cause motor damage, equipment malfunctions and synchronization errors of UPSs and generators during transfers. Generators can be susceptible to high harmonics due to resultant overheating and voltage regulator malfunction.5
  • High-frequency harmonic currents drive up eddy current and proximity losses in distribution transformers and cables.6
  • High-voltage distortion drives up transformer core losses. Higher losses show up as higher energy and cooling costs and increased PUE.7
  • Higher losses lead to higher operational temperatures which are known to reduce the lifetime of transformers and generators.8


OTHER POTENTIAL SYMPTOMS OF HARMONICS

As seen previously, the potential impact of harmonics present on the voltage waveform can manifest in a number of troublesome areas for the data center facility. Some other subtler symptoms may be telling you that your data center has a lurking problem with harmonics. These may include:

  • Flickering and blinking lights
  • Panels, neutral wiring and other distribution equipment running hot
  • Contactors dropping out
  • False breaker tripping
  • Spurious reboot of equipment
  • Unexplainable data errors
  • Load imbalance across three-phase power
  • Load current present on the neutral wire>
  • Poor power factor

According to Green Grid, “For every percentage point that THDi increases, there may be a 2% increase in losses (half from the copper wire, half from the transformers) in the electrical system.”9

Identifying the root cause for any of the symptoms can be a long and tedious manual process if there is not already a granular system of power quality measurement in place across the facility. Making use of highly intelligent rack-level power distribution units enables automation of monitoring and diagnosis, helping staff to be more productive and reducing potential costly downtime.

CIRCUIT BREAKER TRIP FORENSICS

The presence of harmonics can cause false circuit breaker trips. Circuit breakers can also be tripped by excess current flow in a circuit, whether caused by a momentary transient or a long slow climb due to a voltage sag (when voltage goes low, current must go higher to maintain a given power level).

No matter the reason, identifying the underlying cause has been simplified for the data center operator thanks to the circuit breaker trip forensics. This is critical to facilitating troubleshooting efforts by those responsible for maintaining the data center operations.

RECOMMENDATIONS

If you or your team are seeing symptoms of power quality issues in your data center facility, there are a number of steps to take that will help identify the root cause and eliminate the trouble:

Fault isolation: Take steps to localize the source of the problem to a panel and then to a branch. This may mean temporarily turning off one of your utility feeds and exercising your power redundancy circuits.

Choose suitable diagnostic tools: Clamp-on ammeters are good for seeing instantaneous currents/loads but often do not reveal the presence of harmonics. Long-term current monitoring coupled with waveform captures is much more helpful. This may necessitate devoting an oscilloscope to troubleshooting for a time.

Call for help: If your team is unavailable or unable to isolate the cause of a problem, consult with a trusted Legrand partner to look into your situation.

Proactive steps:

  1. Follow IEC 61000-3-2, or newer, recommendations on harmonic emission limits.
  2. Minimize power transmission distances where possible and choose suitable wire gages.
  3. Use active and passive filters where necessary, along with appropriate transformers, to minimize harmonics.
  4. Replace breakers that may have degraded due to being tripped too many times (or used as a power switch).
  5. Work with reputable gear designed for efficiency and quality from trusted suppliers committed to your success.
  6. Perform continuous metering, monitoring, and alarming of harmonics and power quality at a point of common coupling—at the circuit breaker of the in-rack PDU, the panelboard, overhead busway or end-of-row PDU.
  7. A common approach that addresses the adverse effects of harmonics is oversizing, which increases the initial acquisition cost. However, be mindful that oversized equipment will also operate at partial loads leading to lower efficiencies and increased operating costs. Avoid oversizing when you can.
  8. Perform a comprehensive study whenever significant infrastructure changes may make it appropriate to do so.


CONCLUSION

Power quality problems in the data center can lead to faulty equipment (and vice versa). As the data center industry makes strides towards long-term sustainability, improved power efficiency and extending the useful life of infrastructure become very important. Being able to look at detailed power and power quality data on an ongoing basis will help determine whether equipment is operating nominally and affords a chance to predict equipment failures, facilitating the performance of preventive maintenance.

  1. https://en.wikipedia.org/wiki/Electric_power_quality
  2. Green Grid “Data Center Power System Harmonics”
  3. https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf - Ian Wallace
  4. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf
  5. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf
  6. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf
  7. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf
  8. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf
  9. https://3dfs.com/wp-content/uploads/2015/11/WP55_DataCenterPowerSystemHarmonics.pdf
  10. Ibid - https://transcoil.com/wp-content/uploads/2017/12/Harmonic-Distortion-in-Data-Centers.pdf


Download PDF