hIOmon - Benchmarking Support Information

		h	IO	mon	™
		The File I/O Performance Monitor

Making data perform^{^SM}

Home | Products & Services | Contact | About hyperI/O^sm

Big Picture Product Info Features Architecture Screen Shots Documentation Support Download		Benchmarking with hIOmon

		hIOmon is not a benchmarking tool that itself generates disk or file I/O operations (e.g., like Iometer) nor does it perform "system" benchmarking tests that attempt to mimic or simulate application workloads (e.g., PCMark®05 and PCMark Vantage for consumer systems). Rather, hIOmon collects empirical I/O operation performance metrics that reflect the disk and file I/O operation activity actually performed by applications within your own particular computer system environment, and using your very own applications, files, and devices as you normally do within everyday production environments!






Benchmarking Scenarios

Amongst its many uses, hIOmon supports a variety of benchmarking scenarios. As examples, you can use hIOmon to: Perform a "before" and "after" comparison when evaluating proposed improvements, such as the use of Solid State Disk (SSD), faster hard drives, iSCSI, additional cache or system memory, faster/more processors, file defragmentation approaches, virtualized storage, virtual machine consolidation, etc. First gather the empirical I/O operation performance metrics collected by hIOmon (upon a file, device, and/or process/application basis) for your current system and applications of interest, then perform the same collection of metrics after introducing the proposed improvement. Compare the metrics to determine the actual impact of the system change. Understand exactly how your specific applications -- moreover, within production environments -- are actually using their associated files and devices (i.e., an application's particular "I/O profile"), and then compare these application I/O profiles to that of the benchmark. The various device and file I/O operation performance metrics collected by hIOmon can be used to establish a specific application's I/O profile. By using the hIOmon software to also measure the I/O operation activity (e.g., read/write ratio, random/sequential access ratio, etc.) generated by a particular benchmark, you can determine the extent to which this benchmark I/O operation activity coincides with the actual I/O profiles of your particular applications. Along similar lines, benchmarks often attempt to portray the typical I/O activity of typical applications. But are the actual I/O profiles of your particular applications "typical"? The empirical I/O operation performance metrics made available by hIOmon can help you assess how "typical" your applications are and how representative a particular benchmark might be of your specific applications and their I/O usage. Help determine appropriate input parameters (e.g., read/write ratio, queue depth, random/sequential access ratio, etc.) for use with benchmark tools (such as Iometer) that support configurable options related to the type of I/O operation activity that is generated. For instance, by more appropriately configuring benchmarking programs based upon the empirical process-based I/O operation performance metrics provided by hIOmon, you can better represent the actual I/O operation activity of your particular applications. Back to top
Benchmarking Advantages, Metrics, and Interface Support

hIOmon can be a great complement to benchmark testing. The following are just some of the many features that hIOmon provides that can aid you in your benchmarking efforts.

Application Focus

hIOmon enables you to both measure and monitor the performance of your disk storage I/O operations up at the application level and moreover upon an individual, specific file as well as device and process/application basis. The I/O operation metrics collected by hIOmon essentially reflect exactly how the application itself experiences its interaction with the disk storage, as based upon the actual file and disk I/O operations that the application performs. The hIOmon software can measure and monitor an application's I/O operations at three different levels within the operating system (and even concurrently): File Level. By monitoring I/O operations at the file (logical disk) level, hIOmon collects I/O operation performance metrics that reflect the actual experiences of the application itself (e.g., the minimum, average, and maximum read and write I/O operation response times that it actually sees). These metrics also include "system file cache" metrics (where I/O operations performed by the application are satisfied by the system file cache rather than directly by the disk device). Volume Level. This is at the "storage volume" level within the operating system and represents, for instance, a particular partition (e.g., "\Device\HarddiskVolume1") or a "dynamic disk" defined upon a physical disk drive. Physical Device Level. This is at the "physical device" level within the operating system and is akin to the "PhysicalDisk" performance counter level support provided by the Microsoft® Windows® "Performance Monitor (PerfMon)" and "System Monitor" facilities. hIOmon, however, provides additional I/O operation performance metrics, supports both the I/O trace and/or summarized collection and display of the "Physical Device Extended" I/O operation performance metrics, and offers many other advanced features. For examlpe, hIOmon can correlate "physical disk I/O operations" to their respective file I/O operations (i.e., identify to which file the physical device I/O operation is directed), including "split I/O" operation metrics that are often related to fragmented files. With its "application focus", hIOmon moreover lets you view storage I/O operation activity and performance from the application level regardless of the underlying storage (and associated I/O path) infrastructure. In other words, hIOmon measures and monitors an application’s I/O operation activity regardless of the underlying file system type or file placement used, the storage I/O transport used (e.g., SCSI, iSCSI, Fibre Channel), the particular disk storage type/interface used (e.g., SATA, SAS, FC, SSD, RAID-level, etc.), storage volume configuration (or virtualization type) used, etc. Even the impact of system memory used for the system file cache can be evaluated with hIOmon. As a result, you can use hIOmon by itself to "benchmark" the impact of system changes such as those mentioned above. By comparing the empirical disk and file I/O operation performance metrics collected by hIOmon upon a specific file, device, and/or process basis, you can determine the actual extent to which the system change(s) affected the performance of an application's I/O operation activity. Back to top

Metrics

One of the especially significant features of hIOmon is its unique ability to collect, display, and export summarized I/O operation information. These "summary" I/O operation performance metrics (which are automatically aggregated by the hIOmon I/O Monitor software component at the very moment that it actually observes the occurrence of the I/O operations) are an innovative feature of hIOmon that can dramatically reduce the overhead associated with the collection, display, export, and subsequent post-processing of I/O operation performance metrics. hIOmon supports "summary" I/O operation performance metrics upon an individual specific file, device, and process/application basis. Moreover, hIOmon supports a comprehensive set of summary metrics, which include: I/O operation counts and rates (IOPS) I/O operation response times (including minimum, average, and maximum) Data transfer amounts, sizes, and rates (MB/s) Queue depths (including average and maximum) Random/sequential access (including I/O operation counts and data transfer amounts) System file cache metrics (e.g., "hits" and "misses") Associated "physical device" I/O operation metrics (including "split I/O" operation information related to fragmented files) And much more. Altogether, the breadth and depth of the disk and file I/O operation performance metrics provided by hIOmon enable a variety of users to expedite, enhance, and better-focus their benchmarking efforts. Back to top

Interface Support

hIOmon provides a variety of interfaces by which to display and export the disk and file I/O operation performance metrics that it has collected. These interfaces include a Java™-based GUI, a Command-Line-Interface (CLI), Windows Management Instrumentation (WMI) support (including Microsoft Internet Explorer display and sample VBScripts), and Windows Performance/System Monitor support (i.e., Perfmon support). In addition, hIOmon provides extensive CSV Export File support. For example, the hIOmon Manager service can optionally be configured to save/export real-time device and file I/O operation performance information directly to an export file. Moreover, you can optionally specify the particular types of performance metrics that are to be included within the CSV-format "hIOmon Manager Export File"; this option enables you to restrict the contents of the export file to only your particular metrics of interest. The hIOmon Manager can also create an export file for use by the "hIOmon Add-Ons". The various interfaces supported by hIOmon along with its CSV Export File support provide users with flexible and familiar means to display and export the disk I/O operation performance metrics collected by hIOmon, which in turn can help make your benchmarking efforts more productive and pertinent. Back to top

Benchmarking Support Add-On

Included as part of the standard hIOmon software package, the hIOmon "Benchmarking Support Add-On" enables you to quickly and easily utilize the hIOmon software so as to retrieve real-time, user-specified "I/O summary" (i.e., aggregated) operation performance metrics for the files, devices, and processes currently being monitored by the hIOmon I/O Monitor. These metrics can include the "Physical Device Extended Metrics". Moreover, the retrieved metrics can reflect the actual file and device I/O activity generated as a result of running, as examples, a disk and/or file benchmarking tool, a synthetic application workload benchmarking tool, or even applications in everyday usage. This Add-On also allows you to immediately write (export) the retrieved metrics to a Comma-Separated-Values (CSV) disk file. The particular metrics saved to this CSV file are based upon the metric type names that you specify within the header (first) row of the CSV file. This enables you to select only those particular metric types of interest to you (i.e., the CSV file will only include those particular metric types that you specify). You can use this Add-On to retrieve and save the selected I/O operation performance metrics at the completion of your benchmark testing and also periodically during the benchmark testing at your convenience. The Add-On supports the various the various benchmarking scenarios described above. It also includes a configurable hIOmon WMI VBScript that can be used to easily and quickly generate a system report that provides such details as information about the hIOmon software (including hIOmon usage data and details about the various hIOmon output files) along with computer system information (e.g., operating system type and version, basic hardware information, etc.). Please refer to the hIOmon "Benchmarking Support Add-On" description page for additional information about this hIOmon Add-On. Back to top

Where can I learn more about hIOmon?

For additional information about hIOmon, please see the following links: "Big Picture" information about hIOmon and the "5 Storage Performance Questions". Overall product information about hIOmon, including product description, solutions, uses, operating system support, availability and pricing. Feature list and descriptions. Overall of the hIOmon architecture. Screen shots of the hIOmon Presentation Client, CLI support, WMI support, and Windows Performance/System Monitor support. hIOmon "Add-On" support, including the hIOmon "Add-On Support for IBM Tivoli Monitoring" and the hIOmon "Intel NAS Performance Toolkit (NASPT) Support Add-On". hIOmon documentation available for download (including datasheet, readMe document, overview presentation, licensing terms, user guide, and white paper), tutorials, and "best practices". hIOmon support (including FAQs). Information about downloading a full-function trial version of hIOmon.

All trademarks mentioned herein are the property of their respective owners.