Floating Point Precision

[Mike-Purtell]

I have been using the Semi-ATE STDF library for about 1 year, to process STDF files from Advantest 93K SmartTest 8 (java) OS. Works very well for me, could not be happier for having found them.

After the STDF conversion, the test measurement results are stored as float-64. The STDF spec lists measurement results and test limits as float-32. In this sense, measurement results take up twice as much memory as they should, which may slow down the conversion, and lower the maximum STDF file size possible.

I have been optimizing my data analysis scripts by converting all floats from 64 to 32 bits after the STDF conversion, and am seeing impressive performance improvements

My idea/suggestion is for an option to change all of the 64-bit floating point results to 32-bit as early as possible during the STDF conversion. 32-bit floats have similar resolution to 7 decimal digits of resolution, more than enough for my work on RF transceivers ICs. I have no complaints about the speed of the conversion tool, but if this makes it faster, who wouldn't take it? Also, I have had very large stdf files ( > 1G) crash when my PC ran out of memory. Perhaps reducing the size of value allocated to float-64 would allow conversion of larger STDF files.