Data Center Power Distribution:

STARLINE Track Busway, PX™ iPDUs and Power IQ®

Data Center Overview

Data center managers are faced with increasingly challenging demands: supplying additional computing power using less energy in a smaller space while staying within budget constraints and maintaining mission-critical reliability.

These demands often create a culture of frequent and predictable change without compromising the end product. Unfortunately, it can also stress some existing data centers, making them unsustainable for several reasons. These reasons can include: the inability to quickly adapt to changes in computer equipment capacity, the fact that many data centers are nearing or have surpassed the limitations of their power distribution system, and the lack of efficiency due to an outdated or non-existent power monitoring infrastructure. This white paper will address the power distribution and monitoring solutions that have been successful in meeting these demands, and how data centers can be designed to create sustainable IT environments that are capable of satisfying evolving business, financial and regulatory goals.

Power Distribution Trends

Data center managers are looking for ways to increase energy efficiency while providing a highly adaptable IT environment to support service-oriented architectures and rapid changes in demand. Although software, systems , and storage are all becoming highly adaptable, nothing in the data center can adapt quickly unless the power infrastructure is also adaptable.

Problems include the following power-related issues:

• IT equipment is changing with a much higher frequency than the typical life expectancy of the data center. Typical data centers are designed with a life expectancy of 10 to 15 years. However, most IT equipment refresh cycles are 2 to 5 years.
• More powerful equipment often results in an undersized power supply system. The facility design must take a variety of power conditions into consideration:
  • Medium density (5-10 kW/rack) and high density (15-30 kW/rack) equipment
  • Blade servers, storage and high density networking fabrics
  • Single-phase and three-phase power requirements, often mixed within a given rack
  • Physical receptacle requirements
• Corporate and regulatory requirements for energy usage. Beyond the cost of energy, the goals and requirements for Power Usage Effectiveness (PUE), peak demand and carbon footprints must be addressed. A growing component of this system is the ability to measure energy usage on a granular level, driving the energy considerations to the point of use. Smart grid implementation will likely have additional implications for the energy infrastructure.
• Uptime requirements. High levels of uptime are assumed and “table stakes” are generally taken for granted.

The impact of these trends on data center electrical design is obvious: the need for flexible, adaptable, electrical power distribution and monitoring systems.

Traditional Power Solutions: Too Much or Too Little

Traditional data center power distribution designs consist of power distribution units (PDUs ) delivering power to remote power panels (RPPs), which in turn deliver power to racks via “whips” — power cabling routed underneath a raised floor. While such implementations are well known and represent a low risk for the design engineer and electrical contractor, the drawbacks of such designs are well established: valuable floor space is consumed by distribution equipment; raised floors are costly; under-floor cabling becomes congested, disrupting airflow used for cooling; requirements to remove unused cables, which tend to be abandoned; and risk of human error while working with circuit breakers and cables that are not clearly associated with a given load.

Equally important, traditional data center power implementations are neither flexible nor adaptable, and are therefore expensive and time consuming to change. When designing a data center with traditional electrical systems, engineers or designers must pre-plan every outlet — from dedicated hard wired conductors for each power outlet through overhead or raised floors back to the main distribution panels and circuit breakers. It is nearly impossible to predetermine the power requirements for each rack in each location when the data center goes live, let alone plan for future requirements.

As a result, data center managers either overbuild the current system with much higher investment or build a system that will meet the current planned demand, but increase the operating expense when costly changes are required. When inevitable changes occur as a result of growth, equipment upgrades, or changed plans, the costs for electrical changes can be significant, both in terms of labor costs and the risk of outages.

Today’s Power Solutions: Flexible and Adaptable

In recent years, power management solutions have evolved to the point where they are both flexible and easily adaptable to changing requirements. They are also scalable for change and growth and typically provide the lowest long-term cost of ownership.

Solutions include overhead busway systems, intelligent PDUs, and intelligent power management software, working together seamlessly:

• Overhead busway systems are comprised primarily of busway sections (straight lengths containing busbars and a slot for continuous access) and tap boxes (plug-in units containing circuit protection and wiring devices). Systems are available up to 400 amps and 600 volts, AC and DC.
• Intelligent PDUs include switched, non-switched, and metered PDUs and inline meters that provide environment and power metering, monitoring, and management.
• Power management software lets your baseline, monitor, and report energy usage and environmental conditions at any level — data center, rack, department, and equipment type. It also provides centralized monitoring, graceful OS shutdown, outlet control, powerful analytics of your costs, line capacities, carbon footprint and rack temperature.

Overhead Track Busways: Easy, Fast and Inexpensive to Change

The STARLINE® Track Busway system can be tapped at any location with a variety of plug-in units, eliminating panel boards long runs of conduit and wire and expensive installation costs for dedicated power outlets. Dedicated circuit breakers located at the point of use make troubleshooting outages or reconfiguring supplies very easy. The cost to install the STARLINE system is competitive with traditional power distribution systems. Where greater savings come in is when power needs to change. With an overhead bus system, there is no need to work on live panels or scheduled outages to add, move or change outlets. Overhead bus systems eliminate the need to remove and scrap short or undersized cable whips and run new longer or larger ones. Therefore, the risk of unintended potential power outages is avoided. Racks can be installed or moved without disrupting operations. Any location in the data center can have the racks installed, moved, reconfigured, or removed without affecting anything else in the space, and without risking an unplanned outage.

Sustainable

Overhead bus systems can be reusable for years for future facility changes and growth. With the busway, as compared with individual cables, less copper is used and the reusable components save energy and reduce waste for the cost of mining, smelting, and transporting replacement items.

Ever-increasing server densities are causing an increase in kW power density, resulting in increased cooling requirements in today’s data centers. For every kW increase in power, an equal amount of cooling capacity is required. This never-ending cycle of increasing power and cooling requirements translates into more and larger power cables under the floor — stealing valuable cooling space. With an overhead bus system, there is no jungle of wires to obstruct airflow under the floor, making it one of the most energy efficient and safe power systems available in the market today

Scalable

Data center electrical designs are often outdated before they are ever installed. Day one requirements are often different from the original plan, which forces electrical outlet locations and types to be reconfigured, increasing costs and causing schedule delays.

STARLINE is completely scalable, enabling components and power circuits to be added as needed – without tying up capital and wasting resources – rather than completely building out the entire facility in the beginning. This is very beneficial for co-location and other facilities that are built out over time. It also enables a greener data center because changes to increase resource utilization are not hampered by an inability to quickly supply the correct power at a new location.

More Usable Data Center Space

With data center floor space at a premium, every square foot is critical. Overhead bus systems eliminate RPPs, which result in more usable space in the data center for IT equipment and server racks. In addition, miles of power cables are eliminated when power outlets or drops can be located exactly where they are needed.

Monitored Power Usage

It is important for data center managers to have the ability to monitor and know the exact power usage of the data center. All too often, as new equipment is added to a rack, the cable ratings can be exceeded, trip the circuit breakers and cause an unplanned outage. STARLINE Track Busway systems provide power feed monitoring units to measure and display each phase current in real-time, perform alarm functions and provide remote communication. With complete system power monitoring, data center managers can intelligently plan for the future. They know the exact capacity and load of each rack or system and therefore know where they can install new or upgraded equipment. In addition, monitoring helps ensure the electrical system is balanced across phases, leading to better energy efficiency and ultimately costs savings.

400V Intelligent Rack PDUs: Accurate, Intelligent Power Delivery, Monitoring and Management

Maintaining 400V power to the IT equipment rack minimizes transmission losses. Typically, 480V three-phase power enters a data center and is converted to 208V/120V. This conversion causes about a 4% transmission loss. Keeping the power to the rack at 400V means virtually no loss.

Cost Savings

Within the rack PDU, the hot lines are wired to the neutral, resulting in 230V at the rack PDU outlets. (The 400V is the voltage measured between two hot lines, e.g., L1 + L2. Tying a hot line to the neutral, e.g., L1 + N, drops the voltage to 230V {400V/1.732}.) Virtually all modern IT equipment has a universal power supply capable of operating between 100V and 240V, so 230V is a safe operating voltage. In fact, there are additional power savings since operating processors at 120V leads to an 18% loss whereas operating processors at 230V leads to a 15% loss, an efficiency gain of 3%. Taken together, the total net improvement is 7% (4% + 3%). By eliminating unnecessary voltage transformations, 400V power reduces energy costs by approximately 2-3% vs. 208V distribution; and approximately 4-5% vs. 120V distribution.

Power to Spare

Supplying 400V power to each rack minimizes step-down transformation and line transmission loss while simultaneously reducing copper utilization. Typically, 400V three-phase power is connected to two intelligent rack power distribution units (iPDUs) in each cabinet. The rack PDU, with internal line-to-neutral (three-phase Wye) wiring, supplies each server with 240V, single-phase power which is well within the operating range of the power supplies of virtually any IT equipment.

A typical configuration of the rack PDU described above is three input lines and three circuits (L1 + N, L2 + N, and L3 + N). In North America, the National Electrical Code (NEC) requires branch circuits to be protected from overloads at 20A or less. Therefore, each of the three circuits has a 20A circuit breaker (or fuse). Intelligent rack PDUs may monitor the circuit breakers and send alerts when a circuit breaker gets close to tripping. Intelligent rack PDUs may also monitor the three lines and send alerts if the lines become unbalanced. For example, if 20A loads were plugged into circuits #1 (L1 + N) and #2 (L2 + N) but there wasn’t any load in circuit #3, this would leave L3 out of balance with L1 and L2. Such imbalances can overload the neutral wire and cause other problems such as harmonics.

Environmental Monitoring

Intelligent rack PDUs meter power, in some cases at the individual outlet level. Metering can include current, voltage, power (kVA, kW), and consumption (kWh). Ideally, kWh should be measured at +/- 1% billing-grade accuracy. Intelligent rack PDUs can also support environmental monitoring at the rack. Ideally, the rack PDU should support optional plug-and-play environmental sensors, such as temperature and humidity, which don’t require additional network connections or hardware.

The American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) recommends placing temperature sensors on the cool air inlet side of a rack at the bottom, middle, and top. An intelligent rack PDU should also support monitoring temperature with user-adjustable high and low temperature thresholds. When the thresholds are crossed, alerts are sent so the temperature can be lowered if too hot or raised if too low, thereby wasting power by overcooling. Monitoring humidity is also important but since it doesn’t change much with respect to location on a rack, one humidity sensor per rack is sufficient.

Recently, ASHRAE raised the high end of the allowable inlet temperature range to 27° C (81° F). IT equipment vendors ensure that with this inlet temperature, IT equipment will operate safely. By raising the ambient temperature in a data center, IT managers can reduce the energy consumed by air-conditioning.

As a global vendor of rack power distribution units, Raritan provides a very large range of product configurations that accept 400V input. While 400V distribution is relatively new in North America, Raritan has deployed 400V systems in Australia and in many European countries where it is the nominal standard.

Data Center Power Management Software: Powerful Data Collection and Reporting at Your Fingertips

Data center power management software should be able to gather information from a variety of rack PDUs and metered devices since data centers have a variety of meters and vendors. The software should be able to consolidate the information and present it in reports that can be organized to meet the needs of the data center. For example, it might be useful to look at power data organized by data center location or power usage by department or check the efficiency of different servers. If the greenhouse gas cap and trade becomes significant, reports on a data center’s carbon footprint will be useful.

It also makes sense for the data center power management software to gather and manage data center rack environmental information such as temperature and humidity. ASHRAE has established, in cooperation with IT equipment vendors, recommended temperature and humidity ranges for the air at the cool air inlets of IT equipment. The higher the temperature at the cool air input, the less energy is being expended on air-conditioning.

It would also be useful for the power management software to configure rack PDUs. Since a data center may have hundreds of rack PDUs, some way to speed up their configuration would be a welcome feature for most data center managers.

Product Selection

Track busway selection should follow these considerations:

• Continuous access slot design provides the greatest flexibility in plug density and positioning
• Section lengths should be as long as possible (up to 20 feet) to reduce the number of electrical joints
• Tap boxes should have a comprehensive range of ratings and wiring devices
• Tap boxes should have a minimum number of parts and connection points for greatest reliability
• Joints and connection points should be lifetime maintenance-free
• Supplier should have proven track record of performance and delivery

Data Center Power Management Intelligent rack PDU selection should follow these considerations:

• Outlet-level switching so outlets can be turned on and off to control the provisioning of power.
• Unswitched outlets may be desirable for situations and applications where switching off power could present problems, e.g., 24/7 mission-critical servers
• Rich and accurate power data collection including current (A), voltage (V), power (kVA, kW), power factor and energy consumption (kWh) with billing grade +/- 1% accuracy
• Built-in support for plug-and-play environmental sensors
• Buffering of power and environmental data so as not to tax network bandwidth
• Monitor branch circuit breaker current and trip status
• For three-phase power, monitor balance of current draw across the three phases Intelligent

Power management software selection should follow these considerations:

• Multi-vendor management
• User-configurable dashboard
• Clean presentation of information via reports and charts — energy consumption via device, rack or customer
• Monitoring and notifications
• Advanced cooling analytics

Powering the Data Centers of the Future

Change is inevitable, and data centers should be designed with this undeniable precept in mind. Companies that cannot shift with the times or trends because of antiquated technology and infrastructure lose business to more agile competitors. Power management solutions play a fundamental role in implementing more versatile data centers that can quickly evolve to address the demands and challenges of the future.

©2011 Raritan Inc. All rights reserved. Raritan®, Know more. Manage smarter.™, Dominion®, Paragon®, PX™, Power IQ®, and dcTrack™ are registered trademarks or trademarks of Raritan Inc. or its wholly-owned subsidiaries. All others are registered trademarks or trademarks of their respective owners.