At SNW, a couple of weeks back, SNIA annouced the coming out of their green storage initiative’s new SNIA Emerald Program and the first public draft release of their storage power efficiency test specification. Up until now, other than SPC and some pronouncements from EPA there hasn’t been much standardization activity on how to measure storage power efficiency.
SNIA’s Storage Power Efficiency Specification
As such, SNIA felt there was a need for an industry standard on how to measure storage power use. SNIA’s specification supplies a taxonomy for storage systems that can be used to define and categorize various storage systems. Their extensive taxonomy should minimize problems like comparing consumer storage power use against data center storage power use. Also, the specification identifies storage use attributes such as deduplication and thin provisioning or capacity optimization features that can impact power efficiency.
In addition, the specification has two appendices:
- Appendix A specifies the valid power and environmental meters that are to be used to measure power efficiency of the system under test.
- Appendix B specifies the benchmark tool that is used to drive the system under test while its power efficiency is being measured.
Essentially, there are two approved benchmark drivers used to drive IOs in the online storage category Iometer and vdbench both of which are freely available. Iometer has been employed for quite awhile now in vendor benchmarking activity. In contrast, vdbench is a relative newcomer but I have worked with its author, Henk Vandenbergh, over many years now and he is a consummate performance analyst. I look forward to seeing how Henk’s vdbench matures over time.
Given the spec’s taxonomy and the fact that it lists online, near-online, removable media, virtual media and adjunct storage device categories with multiple sub-categories for each, we will focus only on the online family of storage and save the rest for later.
SPC energy efficiency measures
As my readers should recall, the Storage Performance Council (SPC) also has benchmarks that measure energy use with their SPC-1/E and SPC-1C/E reports (see our SPC-1 IOPS per Watt post). The interesting part about SPC-1/E results is that there are definite IOPS levels where storage power use undergoes significant transitions.
One can examine a SPC-1/E Executive Summary report and see power use at various IO intensity levels, i.e., 100%, 95%, 90%, 85%, 80%, 50%, 10% and 0% (or idle) for a storage subsystem under test. SPC summarizes these detail power measurements by defining profiles for “Low”, “Medium” and “Heavy” storage system use. But the devils often in the details and having all the above measurements allows one to calculate whatever activity profile works best for you.
Unfortunately, only a few SPC-1/E reports have been submitted to date and it has yet to take off.
SNIA alternative power efficiency metrics
Enter SNIA’s Emerald program, which is supposed to be an easier and quicker way to measure storage power use. In addition to the specification, SNIA has established a website (see above) to hold SNIA approved storage power efficiency results and a certification program for auditors that can be used to verify vendor power efficiency testing meet all specification requirements.
What’s missing from the present SNIA power efficiency test specification are the following:
- More strict IOPS level definitions – the specification refers to IO intensity but doesn’t provide an adequate definition from my perspective. It says that subsystem response time cannot exceed 30msec and uses this to define 100% IO intensity for the workloads. However given this definition it could apply to random read, random write, or mixed workloads and there is no separate specification for sequential or random (and/or mixed) workloads. This could be tightened up
- More IO intensity levels measured – the specification calls for power measurements at an IO intensity of 100% for all workloads and 25% for 70:30 R:W workloads for online storage. However we would be more interested in also seeing 80% and 10%. From a user perspective, 80% probably represents a heavy sustainable IO workload and 10% looks like a complete cache hit workload. We would only measure these levels for the “Mixed workload” so as to minimize effort.
- More write activity in “Mixed workloads” – the specification defines mixed workload as 70% read and 30% write random IO activity. Given today’s O/S propensity to buffer read data, it would seem more prudent to use a 50:50 Read to Write mix.
Probably other items need more work as well, such as defining a standardized reporting format containing a detailed description of HW and SW of system under test, benchmark driver HW and SW, table for reporting all power efficiency metrics and inclusion of full benchmark report including input parameter specifications and all outputs, etc. but these are nits.
Finally, SNIA’s specification goes into much detail about capacity optimization testing which includes things like compression, deduplication, thin provisioning, delta-snapshotting, etc. with an intent to measure storage system power use when utilizing these capabilities. This is a significant and complex undertaking to define how each of these storage features will be configured and used during power measurement testing. Although SNIA should be commended for their efforts here, this seems to much to take on at the start. We suggest capacity optimization testing definitions should be deferred to a later release and focus now on the more standard storage power efficiency measurements.
—-
I critique specifications at my peril. Being wrong in the past has caused me to re-double efforts to insure a correct interpretation of any specification. However, if there’s something I have misconstrued or missed here that are worthy of note please feel free to comment.