Why use IEEEIO?

Its compact
On the SGI Origin 2000, the entire library is 206k. It contains only a little over 2,500 lines of code for the C++, C, and F77 versions combined.

Its fast
Most simulation applications end up writing a lot more data than they read, the I/O interface is optimized for maximum writing speed. No seeking is required during writing and the low-level raw unix I/O interface is used to maximize performance. Preliminary performance tests show that it is comparable with f77 unformatted IO for writing speed. Data is translated to the machines native representation (ie. byte-swapping) only during reading. If speed is required for reading as well, then there is a 'convert2native' utility which will optimize the file for the type of machine that is reading it.

It interoperates with HDF
If you use HDF-SDS, you should have no problem using IEEEIO because they share the same programming interface. You indicate the type of file you want to use when you open the file. From there on, the same subroutines are used for reading or writing data to the file, whether it be HDF or IEEEIO. In addition there is an 'ioconvert' utility that can translate either direction between IEEEIO and HDF files and preserve all of the HDF annotations and NetCDF attributes.

IEEEIO doesn't supercede HDF. It is designed to be a lightweight IO system which you would use when the broader functionality of HDF is not required.

It is crash-safe
Since no seeking is required to maintain file integrity during writes, you will not lose data if the simulation program exits without having properly closed the file.

Most machines use IEEE standard floating point number formats
When HDF and XDR were first created, almost every machine architecture had its own native floating point representation. There were VAX floats, Cray floats, and Convex floating point representations... all mutually incompatible. Now, virtually every major supercomputer architecture supports IEEE standard floating number representations. The primary differences between number formats now it the byte-order which is trival to convert. As a result, the XDR and HDF support for platform neutral floating point formats is more work than is necessary for most machines. By depending on IEEE standard number formats, the data storage can be made much simpler and more efficient. If the machine you use doesn't support IEEE floating point numbers, then you should continue to use HDF. Since IEEEIO interoperates with HDF, this does not require significant modifications to your code to switch back and forth between the two. Currently IEEEIO has been ported to SGI IRIX {5.x,6.x}, IRIX64 6.x, HP-UX 9.01, ConvexOS {on SPP-1200 and SPP-2000}, DEC Alpha OSF/1, and the Cray T3E. The only machine that is not supported is the Cray C90.

Why not use XDR?
So if the aim is to create a lightweight portable binary file format, this leads to the question of why not use XDR for the encoding? Well the answer is that in early testing it was discovered that HDF is significantly faster and more efficient than XDR. So if you really need XDR then you should be asking yourself why not use HDF?

Again, this file format is primarily designed to take advantage of the fact that most of the operations that XDR and HDF do to make the data platform neutral are not always necessary given current binary formats for floating point numbers. It also leads to a very simple-to-parse file structure which brings us to the next point...

Simple file structure
Its extremely simple file structure makes it easy to build readers in a variety of languages. This is important for interpreted data processing languages like IDL because the reader can be implemented in native IDL instead of writing native methods. The same is true of Java. Instead of creating native methods that directly call the libieeeio interfaces, the format is simple enough that it can be implemented using java itself.


John Shalf
Last modified: Tue May 27 18:14:25 CDT 1997