h

IO

mon

™

  The File I/O Performance Monitor

Home  |  Products & Services  |  Contact  |  About hyperI/O^sm

Big Picture

hIOmon Fragmented File I/O Operation Metrics Analysis
 Add-On Support

Product Info

The hIOmon File I/O Performance Monitor enables you to collect "split I/O" operation metrics that represent actual file I/O operation activity to fragmented files.  The hIOmon User Guide document provides specific information about the requirements and procedures for collecting, displaying, and exporting "fragmented file I/O operation" metrics, including "step-by-step" instructions using the hIOmon Presentation Client.

Features

Architecture

Screen Shots

Documentation

Download

The hIOmon "Fragmented File I/O Operation Metrics Analysis Add-On" is included within the hIOmon File I/O Performance Monitor software package.  The Add-On itself makes use of two main components that can help you easily and quickly collect, export, and display a set of "fragmented file I/O operation" related metrics:

• The "hIOmonAMCSS" script file.  This "hIOmon Automated Monitoring Configuration Setup" VBScript provides a "Fragmented File I/O Operation Metrics Analysis" option that can be used to automatically configure the hIOmon File I/O Performance Monitor software to collect for export a specific set of "fragmented file I/O operation" performance metrics.  The hIOmon software is configured so that these metrics are collected either for those particular logical disks (e.g., the "C", "E", etc. lettered drives, for overall device-based metrics) or for those particular files (e.g., "C:\Program Files\ThisApp*, for metrics limited to the files indicated) that you specify as input to prompts displayed by the script.  

In addition, the script configures the hIOmon software to automatically export (save) a specific set of collected metrics to a "hIOmon Manager Export File", which is a Comma-Separated-Values (CSV) file that can then be imported into Microsoft® Office Excel spreadsheet, a database, etc. for subsequent analysis.

• Microsoft Office Excel template/macros.  These hIOmon Excel template/macros can be used to directly import the "fragmented file I/O operation" performance metrics from the "hIOmon Manager Export File" noted above and to easily generate a variety of graphs/charts based upon these metrics.  For example, you can graph upon a logical disk basis over a period of time the maximum number of "split I/O operations" that were incurred by a single file I/O operation.

NOTE:

The "fragmented file I/O operation" performance metrics that are collected, exported, and displayed through the use of these hIOmon utilities reflect the actual file I/O operations performed through the normal, everyday use of your very own applications and associated files!  And no file, application, or operating system changes are required!  

The configuration script along with the hIOmon Excel template/macros provide a quick and easy way to help you evaluate and understand the specific performance impact of file fragmentation, once again based upon empirical metrics directly related to the individual files that you are actually using.  Additional information is provided below (and also within the "hIOmon Add-On User Guide" document).

Online tutorials for both the "specific-device" and the "specific-file" configuration options are also available.

hIOmon Configuration Setup for Fragmented File I/O Operation Metrics

The hIOmon “Automated Monitoring Configuration Setup” VBScript provides a specific “Fragmented File I/O Performance Metrics” option that uses the hIOmon WMI Support to help automatically define and activate a Filter Selection (that identifies which particular logical and physical disks are to be monitored by hIOmon) and to automatically create and activate a "hIOmon Manager Export File" to which a specific set of "fragmented file I/O operation" performance metrics are exported (saved) for subsequent use by the hIOmon Excel template/macros.

1. That the hIOmon File I/O Performance Monitor software be properly installed and active (i.e., that the hIOmon Manager and hIOmon I/O Monitor components are currently running; note that these components are normally configured upon the installation of the hIOmon software to automatically be started when the computer system is started).

2. That both the "Summarized Data" and "Raw Device Extended Feature" options are licensed.  Note that the trial version of the hIOmon File I/O Performance Monitor software includes full-support for both of these licensing options.

3. This script is interactive; as such it requires limited user input (e.g., the specific drive letters for those particular Logical Disks to be monitored, or the file path name for those particular files to be monitored) as shown below

The "hIOmonAMCSS.vbs" VBScript file can be found in the "VBScript" folder within the "..\hIOmon\Support\hIOmonWMI\Scripting\Wsh" directory where the hIOmon software was installed.

The script can also be run by means of the "Start -> Programs -> hIOmon -> Getting Started Help -> hIOmon Automated Monitoring Configuration Setup" shortcut (that was added to the Start Menu by the Windows Installer as part of the hIOmon software installation process) or optionally run as part of the Windows Installer installation process for the hIOmon software.

To run the script as part of the Windows Installer installation process for the hIOmon software, simply check the "Invoke the hIOmon Automated Monitoring Configuration Setup Script" checkbox on the "Custom Installation Options" display that is shown as part of the installation process.

To run the script anytime after the hIOmon software has been installed, simply double-click upon the script file (or the shortcut above), which will then display a prompt requesting the particular configuration option to be performed. Select the "Fragmented File I/O Performance Metrics" option as shown below, then click on the "OK" button:

The configuration script will subsequently display the following prompt asking whether you want to collect metrics upon an overall device basis (configuration option 1, "Specify devices") or limited to specific files (configuration option 2, "Specify particular files"):

 Monitoring specific DEVICES for overall device metrics 

If option 1 is selected (i.e., "Specify devices"), then the configuration script will display a prompt requesting the letter of a Logical Disk to be monitored by the hIOmon software:

Click on the "OK" button to add the specified Logical Disk to those Logical Disks for which the hIOmon software is to collect "fragmented file I/O operation" performance metrics. 

Enter a single Logical Disk letter (e.g., "C", "E", "F", etc.) for each click of the "OK" button.  Please be sure not to enter any CD-ROM, DVD, or floppy disk drives (since such device types are not monitored by the hIOmon I/O Monitor).

Also please note that:

• This script and configuration option can be used during the installation of the hIOmon software (or anytime thereafter).

• The hIOmon software will monitor all files upon the specified Logical Disk.

• "Fragmented file I/O operation" performance metrics will be collected for all of the monitored files upon a "summary" basis for each individual monitored file.  These metrics will also be collected for the associated Logical Device in aggregate (i.e., upon a "Device Summary" basis, which reflects the accumulated I/O operation performance metrics for all monitored files resident upon the same disk).

• "Physical Device Extended Metrics" will also be collected for all physical disk volumes that are associated with the Logical Disks specified to be monitored by hIOmon.  These metrics basically reflect "physical device" I/O operations (including "split I/O" operations) that were required to complete file I/O operations for those files that were also being monitored by the hIOmon I/O Monitor.  In addition, "Device Summary" metrics will be collected for the physical disk volumes being monitored by hIOmon. 

• The "summarized" metrics will be collected by the hIOmon I/O Monitor for offload/export upon a periodic basis (by default, every 10 minutes for individual files, and every hour for devices; these default time period values can easily be changed by simply modifying the "default variables" within the script).  Note, however, that by default only "Device Summary" metrics will actually be offloaded to the export file (i.e., summary metrics for the individual files monitored are not offloaded/exported).

• The configuration script will automatically exclude the "..\hIOmon\Support\hIOmonGr" directory (where the hIOmon software was installed) from being monitored by hIOmon if this directory resides upon a Logical Disk that is being monitored by hIOmon.

Click on the "Cancel" button after you have entered all of the Logical Disks that you want monitored by hIOmon.  The configuration script will automatically create a Filter Selection as described below. 

Monitoring particular FILES for file-specific metrics

If option 2 is selected (i.e., "Specify particular files") in response to the configuration setup prompt above, then the configuration script will issue several prompts that allow you to specify the particular files that are to be be monitored by hIOmon. 

Simply indicate:

1. The full path name of the particular file (or files using the wildcard) to be monitored by hIOmon.

2. The time period interval to be used for the collection of the summary I/O operation performance metrics.

Also please note that:

• This script and configuration option can be used during the installation of the hIOmon software (or anytime thereafter).

• The hIOmon software will only monitor only those particular files that you have specified to be monitored by hIOmon.

• "Fragmented file I/O operation" performance metrics will be collected for all of the monitored files upon a "summary" basis for each individual monitored file.  "Device Summary" metrics for the associated Logical Device will not be collected.

• "Physical Device Extended Metrics" will also be collected for all physical disk volumes that are associated with the files specified to be monitored by hIOmon.  These metrics basically reflect “physical device” I/O operations (including "split I/O" operations) that were required to complete file I/O operations for those files that were also being monitored by the hIOmon I/O Monitor.  "Device Summary" metrics will not be collected for the physical disk volumes being monitored by hIOmon. 

• The "summarized" metrics will be collected by the hIOmon I/O Monitor for offload/export upon the periodic basis that was explicitly specified by means of the configuration script prompts noted above.

• The configuration script will automatically exclude the "..\hIOmon\Support\hIOmonGr" directory (where the hIOmon software was installed) from being monitored by hIOmon if this directory resides upon a Logical Disk that is being monitored by hIOmon.

Click on the "Cancel" button after you entered all of the files that you want monitored by hIOmon.

Automated Filter Selection creation

Once you have entered all of the Logical Disks (with option 1, "Specify devices") or particular files (with option 2, "Specify particular files") that you want monitored by hIOmon, the script will automatically create a Filter Selection (named "FragIOstatsFS" by default), which identifies the specific files/disks for which I/O operation performance metrics are to be collected by hIOmon.  The script will also make this Filter Selection the "default" Filter Selection (so that it will automatically be used each time the hIOmon Manager service is started as part of starting your computer).

If the "physical disk volumes" (that are associated with the Logical Disks or particular files specified to be monitored by hIOmon) are not currently attached to the hIOmon I/O Monitor component (this attachment is performed as part of the computer system boot startup process), then the computer system will need to be restarted before the hIOmon software can collect the I/O operation performance metrics as requested within the Filter Selection.  In this case, the script will display the message below (note, however, that the system will not automatically be restarted; clicking on the "OK" button simply closes the message display):

In the event that a system reboot is not required, the script will automatically activate the Filter Selection (so that the hIOmon I/O Monitor will immediately begin collecting metrics for those files/devices specified within the Filter Selection) and then display the following message: 

The script will also automatically create a "hIOmon Manager Export File" (named "FragIOstatsExportFile" by default) that is to include the particular I/O operation performance metric types for which collected metrics are to be exported/saved.  The particular metric types to be exported are noted below.

The script will also automatically activate/start this hIOmon Manager Export File so that the hIOmon Manager can immediately begin exporting the collected I/O operation performance metrics to this export file.  The following message will be displayed by the script to confirm that exporting has been activated.  In addition, the hIOmon Manager Export File will be made a "default" export file (so that it will automatically be used each time the hIOmon Manager service is started as part of starting the computer system).  

That's it!  Simply enter the letters of the Logical Disks (or particular files) for which you want the hIOmon software to collect I/O operation performance metrics (and reboot the system if necessary).  The configuration script automatically configures the hIOmon software as required based upon the Logical Disk letters (or particular files) that you specify.

As described below, this script can also be subsequently used to change the Filter Selection to include additional Logical Disks (or particular files) to be monitored by hIOmon.  The hIOmon "Excel Template/Macros" description provides information on how you can import the "fragmented file I/O operation" performance metrics contained with the hIOmon Manager Export File into an Excel worksheet and generate graphs/charts based upon these metrics.

Back to Top     

Simply double-click upon the script file (or on the shortcut above) to run the script again.  Since the Filter Selection will have been created when the script was previously executed, the following message will be displayed if you had previously selected option 1 (“Specify devices”) in response to the prior Devices/Files prompt:   

To add another Logical Disk letter to the existing Filter Selection, simply click on the "Yes" button, which will cause the script to display the "Logical Disk letter" prompt for input (as previously shown above).  Note that if you had selected option 2 ("Specify particular files") in response to the prior Devices/Files prompt, then you will be asked whether you want to add another file path name to the existing Filter Selection.

To start with a new "FragIOstatsFS" Filter Selection, click on the "No" button, which will cause the script to delete the existing Filter Selection and then display the "Logical Disk letter" prompt (same as that shown above if configuration option 1, "Specify devices", was selected) or the "file path name" prompt (if configuration option 2, "Specify particular files", was selected) for input to be included within a new Filter Selection.

Once you have completed modifying the Filter Selection, the script will then again determine whether the "physical disk volumes" (that are associated with the Logical Disks or particular files specified to be monitored by hIOmon) are currently attached to the hIOmon I/O Monitor component.  If not, the script will display the message saying that a system reboot is required.

Click on the "Cancel" button to immediately end the script, in which case no hIOmon software configuration changes will be made.  

After you are done modifying the Filter Selection, the script will then try to create the hIOmon Manager Export File. The script will display the following message if it detects an existing hIOmon Manager Export File with the same name:

Click on the "OK" button to delete the existing hIOmon Manager Export File.  After this hIOmon Manager Export File has been deleted, the script will then create a new hIOmon Manager Export File (with the same name), which will then be activated and made a default by the script.

The script will subsequently display messages to indicate that the modified/new Filter Selection has been activated/loaded and that the new hIOmon Manager Export File has also been activated.

The script automatically configures the hIOmon Manager Export File to include the following I/O operation performance metric types:

• Name of the monitored device if the "Specify devices" configuration option was selected, otherwise the name of the monitored file

• Timestamp (month, date, hour, minute) when the metrics were prepared for offload to the hIOmon Manager service (so that they could then be exported/saved to the export file); the "second" will also be included if you selected the "Specify particular files" configuration option and also requested a period time interval of less than one minute.

• Separately for both read and write I/O operations and as observed by the hIOmon I/O Monitor:
  -  Accumulated number of I/O operations
  -  Average queue length (i.e., "outstanding I/O operations")
  -  Maximum queue length
  -  Accumulated total (in seconds) of the time durations associated with the I/O operations
  -  Maximum time duration (in seconds) for an observed I/O operation (i.e., maximum response time)
  -  Accumulated total (in bytes) of data transferred by the I/O operations
  - Largest data transfer size (in bytes) associated with an observed I/O operation (i.e., maximum data transfer size)
  -  Accumulated number of I/O operations that represent random access (i.e., the random access I/O count).  An I/O operation is considered to be a random access when the data transferred at its starting address is not contiguous to the address of the last data transferred by the preceding I/O operation.
  -  Accumulated total (in bytes) of data transferred by I/O operations that performed a random access (i.e., the random access data transferred amount)
  -  Accumulated number of I/O operations that represent sequential access (i.e., the sequential access I/O count).  An I/O operation is considered to be a sequential access when the data transferred at its starting address is contiguous to the address of the last data transferred by the preceding I/O operation.
  -  Accumulated total (in bytes) of data transferred by I/O operations that performed a random access (i.e., the sequential access data transferred amount)

• Average and maximum queue length for read and write I/O operations combined

• Separately for Physical Device read and write I/O operations and as observed by the hIOmon I/O Monitor:
  -  Accumulated number of I/O operations
  -  Accumulated number of Split I/O operations
  -  Accumulated number of file I/O operations associated with Physical Device I/O operations that were "split I/O" operations; this count basically reflects the number of file I/O operations that incurred two or more Physical Device I/O operations.
  -  Maximum number of Physical Device "split I/O" operations observed for a single file I/O operation
  -  Accumulated  total (in seconds) of the time durations associated with the I/O operations
  -  Maximum data transfer size (in bytes) associated with an observed I/O operation
  -  Accumulated number of I/O operations that represent random access (i.e., the random access I/O count).  An I/O operation is considered to be a random access when the data transferred at its starting address is not contiguous to the address of the last data transferred by the preceding I/O operation.
  -  Accumulated total (in bytes) of data transferred by I/O operations that performed a random access (i.e., the random access data transferred amount)
  -  Accumulated number of I/O operations that represent sequential access (i.e., the sequential access I/O count).  An I/O operation is considered to be a sequential access when the data transferred at its starting address is contiguous to the address of the last data transferred by the preceding I/O operation.
  -  Accumulated total (in bytes) of data transferred by I/O operations that performed a sequential access (i.e., the sequential access data transferred amount)

• Duration (i.e., the length of time in seconds during which the associated summary I/O operation performance metrics were collected by the hIOmon I/O Monitor)

• Idle time total (i.e., accumulated total duration in seconds as observed by the hIOmon I/O Monitor in which there were neither I/O operations in process nor any I/O operations queued)

• Statistics Time Duration Base (i.e., time duration in seconds since the hIOmon I/O Monitor first started accumulating the associated summary I/O operation performance metrics)

Please see "Appendix A - Export Data Definitions" in the hIOmon User Guide document for additional information about the metrics above.

Also note that if the "Specify devices" configuration option was selected, then the script automatically configures the hIOmon Manager Export File such that this export file contains only "Device Summary" metrics (that is, summary metrics for the individual files monitored are not exported; only the "Device Summary" metrics, which represent the aggregate for all of the monitored files associated with a particular disk, are written to the export file).

If instead the "Specify particular files" configuration option was selected, then the configuration script will automatically configure the hIOmon Manager Export File such that this export file contains only summary metrics for those particular files specified to be monitored by hIOmon.

Back to Top

hIOmon Excel Template/Macros to Import/Graph Fragmented File I/O Operation Metrics

The script described above configures the hIOmon software to automatically export (save) a specific set of collected I/O operation performance metrics to a hIOmon Manager Export File.  The hIOmon "Device and File I/O Performance Analysis Add-On" includes a hIOmon Excel template and accompanying macros that can also be used by the hIOmon "Fragmented File I/O Operation Metrics Analysis Add-On" to directly import these metrics from the hIOmon Manager Export File into an Excel worksheet and to easily generate a variety of graphs/charts based upon the imported metrics.

Please refer to the hIOmon Excel Template/Macro description page for specific information about using the hIOmon Excel template/macros.

Back to Top

hyperI/O^sm -  improving the performance of storage I/O in computer servers