Industry Tutorials
2014 – Synthetic Application Workloads
2015 – How Fast is Fast? Block IO Performance on a RAM Disk
Posted in: Company News, SNIA SSSI, Testing
Leave a Comment (0) →Summary Performance Comparison – HDD, SSHD, SSD
This below is an excerpt of a recent white paper authored by Calypso Systems CEO, Eden Kim and published by SNIA SSSI. The below measurements were taken by Calypso Systems, Inc. on the SSSI RTP/CTS test platform pursuant to SNIA Solid State Storage Performance Test Specification version 1.1.
How does one use the PTS to evaluate and compare SSD performance? First, data should be collected by using the SNIA PTS on an SSSI Reference Test Platform (RTP) to get standardized and normalized results. The user should then select the key measurements from each of the three metrics (IOPS, TP and LAT) to compare the overall performance.
There are commonly accepted block size and RW mixes that are used to evaluate SSD performance. While users may have an interest in other specific access patterns, industry convention uses device level “corner case” (or boundary) testing to evaluate SSD performance (see Appendix – Test Practices).
While IOPS, TP and LAT have long been used for the testing of HDDs, the SNIA PTS methodology has added solid state storage specific processes to accommodate the uniqueness of SSD performance. Specifically, the PTS defines a standardized Pre-conditioning and Steady State determination to ensure the SSD is properly prepared as well as SSD specific tests that address SSD specific performance behaviors.
The chart above shows summary performance data for HDD, Solid State Hybrid Drive (SSHD), mSATA, SATA, SAS and PCIe SSDs. In this chart, there are four measurements for IOPS, two for TP and two for Response Times. This provides an overview of overall performance. Of course, different access patterns can be selected for summary view.
Click here to go to the SNIA SSSI SSD Performance – A Primer White Paper
Click here to go to the SNIA SSSI Tech Notes – The PTS User Guide
SNIA Press Release for PTS-E 1.1 – Enterprise Spec
SNIA announced the update to the Enterprise Performance Test Specification (PTS-E 1.1). The draft is available for public review at www.snia.org/publicreview.
SNIA SSSI also announced an updated Reference Test Platform (RTP) www.snia.org/forums/sssi/rtp and a recommends testing services thru their certified PTS testing facility www.snia.org/forums/sssi/ptstest – Calypso Systems, Inc.
Article found here: http://finance.yahoo.com/news/version-enterprise-ssd-performance-test-211137225.html
Posted in: Company News, SNIA SSSI, Testing
Leave a Comment (0) →“Up To…” “Sustained…” & “Steady State” SSD Performance
Much industry effort has been expended to define SSD Performance, and more specifically, the settling of Performance metrics over time and use. However, SSD marketing claims of “Up to XXX IOPS…” or “Sustained IOPS of XXX” continue to permeate the industry.
Marketing claims of “Up to XXX IOPS…” is synonymous with “Fresh-Out-of-Box” (or “FOB”) performance. This FOB metric has been clearly shown to be extremely transitory (out run in a matter of minutes or a few GB of writes). Further, FOB IOPS are a “peak” value which is not achieved after garbage collection and TRIM. At best, IOPS performance will achieve only some portion of the initial FOB IOPS measured immediately after a Purge of the SSD.
“Sustained IOPS of XXX” is a measure of IOPS performance after a pre determined type and amount of pre writes before IOPS measurements are taken. However, this “Sustained IOPS” value is highly dependent – and will vary – on the type and amount of pre writes (or preconditioning) to which the SSD is subjected.
Use of the “Steady State” IOPS value provides a more consistent and reliable measure of comparative IOPS performance. Using a pre defined and repeatable preconditioning ensures that the SSD IOPS measurements will be accurate, repeatable and reflective of IOPS performance as would be seen after normal use of the SSD.
One of the most reliable preconditioning methodologies is the Steady State preconditioning set forth in the SNIA Solid State Storage Performance Test Specification (“PTS”). This Steady State preconditioning methodology has been carefully developed and vetted by the SNIA Solid State Storage Technical Working Group – an association of over 54 leading SSD OEMs and Controller OEMs. The full PTS can be accessed in the following link.
Posted in: Testing
Leave a Comment (0) →The IO Stack
How IOs traverse the SW/HW stack and are ultimately “seen” by the SSD at the physical Device level is important to understand when testing SSDs.
File Systems, System Caches and Device Drivers all influence how the IOs are presented to the SSD and returned to Application space. While a user application may create a certain Access Pattern of IOs, these IOs may be subject to System File Caching or File Fragmentation.
IOs can be split, concantonated or coalesced. The timing of IOs can be influenced by “Read Ahead Pre Fetches” and “Write Behind Lazy Write” Cached IOs resulting in a loss of one-to-one correspondence of the original IO to the IO at the SSD level. For example, a large block 1024KiB SEQ write – perhaps from a streaming video – could be split by the OS into concurrent 128KiB Writes, effectively creating RND 128KiB Writes to the SSD. Alternatively, a stream of SEQ 4KiB Writes could be coalesced by the system into a large block SEQ Write. Finally, small block RND 4KiB Read Writes may be written to system cache and never make it to the SSD Device.
The net impact is that IOs at the SSD Device level can be very different from what came from, or ultimately ends up returning to, the System level user mode. Accordingly, a test sponsor should be careful in selecting the test software or benchmarking tools.
CTS is a Linux based, HBA Device level software tool that applies a known and repeatable test stimulus directly to the SSD device. Used in conjunction with a RTP Hardware platform, RTP / CTS can normalize effects of the hardware and help isolate performance measurements to the SSD Device.
Posted in: Testing
Leave a Comment (0) →SNIA PTS 1.1 Tutorial – Easen Ho
Dr. Easen Ho tutorial previews PTS-E 1.1 test methodologies during the 2012 Fall SNW at the Santa Clara Convention Center, Santa Clara, CA.
SNIA SSSI PCIe Taskforce – May 2012
Kickoff of new Task Force.
Posted in: SNIA SSSI
Leave a Comment (0) →SNIA SSSI PCIe Round Table – January 2012, Santa Clara, CA
SNIA kicks off the new PCIe Task Force in Santa Clara January 2012
Posted in: SNIA SSSI
Leave a Comment (0) →Engineering TV – Bill Wong Interview, Storage Visions 2012
Bill Wong interviews Eden Kim to discuss the SNIA Solid State Storage Performance Test Specification at Storage Visions 2012.
