The Key Metrics for Maximizing Energy Efficiency in Data Centers
The cost for electrical energy represents 25-40 percent of the operational expenditures in data centers today. Both the US Department of Energy and Gartner have observed that the cost to power a server over its useful life can now exceed the original capital expenditure. Gartner also notes that it can cost over $50,000 per year to power a single rack of servers.
This level of expenditure might be acceptable if all that energy were being put to useful work. But that is not the case, as a considerable amount of it—more than half in most data centers—continues to be wasted, with the major cause being underutilized servers. For example, a 150 Watt server with a utilization of only 10 percent (typical of dedicated servers) has an energy efficiency of only 5 percent (15 Watts worth of useful work for the total of 300 Watts needed to power and cool the server).
The extent of this waste has motivated organizations to begin monitoring and managing data center power utilization more effectively. Getting the most benefit from these initiatives involves maximizing Work/Watt with highly-utilized, energy-efficient servers. High utilization is critical because much of the power being consumed by idle and underutilized servers is wasting energy. But high utilization of inefficient servers also wastes energy, so it is equally important to use more energy-efficient models.
This article builds on a previous article tilted Achieving Energy Proportionality in Data Centers that outlined four steps to minimizing waste. This article examines the metrics needed to maximize Work/Watt.
The Metrics that Matter when Maximizing Work/Watt
The two power consumption metrics that matter the most for servers are: Transactions per second per Watt (TPS/Watt) and Watts Peak (WP). These two metrics together provide what is needed to prevent Watts from being wasted by underutilized and/or inefficient servers, or from being stranded and, therefore, unavailable to be put to work.
The previous article explained why the ENERGY STAR rating system was of little practical value to IT managers. The EPA recently introduced an enhanced efficiency rating system: ENERGY STAR for Computer Servers v2. This new version uses the Server Efficiency Rating Tool (SERT)™ from the Standard Performance Evaluation Corporation (spec). SERT produces a standard Power and Performance Data Sheet that now takes into account power consumption at various workloads, from idle (0%) to 100%. SERT also provides a rudimentary Power-Performance Ratio that measures operations/Watt (ops/W).
While these are definite improvements over the original ENERGY STAR rating system, the results are difficult to interpret and use. For example, the server-side Java operations will not fully exercise some CPUs, resulting in at best “apples and oranges” comparisons among different vintages and models of servers. And SERT does not even support servers with more than 4 processor sockets, which are likely to be used in the most energy-efficient servers.
The PAR4 Energy Efficiency Rating System was also introduced in the previous article. PAR4 is a UL-approved (Underwriters Laboratories 2640) standard testing methodology for accurately measuring server power consumption and energy efficiency. PAR4 provides a critical metric that the enhanced ENERGY STAR rating system does not: a normalized calculation based on TPS/Watt that fully exercises the server’s CPU(s) and takes into account performance advances according to Moore’s Law. This gives IT managers a simple and accurate way to compare various vintages and models of servers during refresh cycles and when expanding capacity.
The PAR4 Method
The PAR4 testing methodology measures four states of server power consumption: when the equipment is Off (plugged into an active outlet but not powered on), Idle (running the operating system but not any applications), Loaded (running at 100% capacity) and at Peak (the highest measured amount of power consumption).
This PAR4 methodology overcomes the limitations and eliminates the vagaries of other energy efficiency metrics to provide more accurate and actionable results. For example, all tests are performed with identical software that prevents altering the environment or test cycle in ways that might produce “more favorable” results.
The table below shows sample results from a PAR4 test. An important hallmark of a valid test is that Peak power consumption (WP) occurs when the server is fully (100%) loaded, which is often the case when booting. PAR4 assures this by utilizing all of the CPU’s threads (for all single- and multi-threaded cores) during the test. Fully utilizing the CPU is also the only way to ensure that the TPS/Watt calculations are fully valid for maximizing Work/Watt.
The PAR4 Power Consumption Details table summarizes the power, voltage, amperage and power factor for all four modes of operation tested.
The PAR4 Metrics
The most useful metric in maximizing Work/Watt is a server’s TPS/Watt. As integrated circuit densities increase under Moore’s Law, the performance of CPUs roughly doubles every two years with relatively little increase in power consumption. The result is a steady improvement in TPS/Watt with every new generation of processors.
To take into account the effect of Moore’s Law, the PAR4 rating system uses a base year of 2000 with processor performance doubling every two years. The graph below shows the PAR4 rating for a server introduced in 2011. This particular server remains an energy-efficient model, so it might not be a good candidate for replacement during a refresh cycle. Note that because the PAR4 rating is logarithmic (base 2), every 100-point increase indicates a doubling of efficiency.
The PAR4 rating takes into account the depreciation of server energy efficiency over time based on Moore’s Law.
PAR4 is the only energy efficiency rating system capable of normalizing TPS/Watt over time, and that is critical to maximizing Work/Watt in the most cost-effective manner during refresh cycles and when adding capacity. And because the PAR4 benchmark test fully exercises the server’s CPU(s), IT managers can depend on its TPS/Watt and WP metrics when pursuing Software-Defined Power initiatives to increase overall reliability and reduce waste.
PAR4 overcomes the limitations of both the original and the enhanced ENERGY STAR rating systems.
Clemens Pfeiffer is the CTO of Power Assure and is a 25-year veteran of the software industry, where he has held leadership roles in process modeling and automation, software architecture and database design, and data center management and optimization technologies.
- Increase the Value of Demand Response Through Automation
- NAEM Research Report: Planning for a Sustainable Future
- NAEM 2015 EHS and Sustainability Software Buyers Guide
- 2015 Environmental Leader Product & Project Awards
- Improve Your Company's Environment and Energy Performance
- Unlocking the Value of Energy & Operational Data
- Get Smarter About Your Energy Procurement Data Book
- Best Practices in Electricity Procurement
- Let's Do The Math for DR
- How "Fixed" is the Fixed Price Product?