There's a post about it here:
http://forum.efilive.com/showthread.php?t=792
Quote:
It is due to the precision of the data stored in the PCM.
The PCM stores each single cell's value as a 16 bit number.
That means each value can only have 1 of 65536 distinct integer values.
For example (assuming the precision is 1/64 per bit - it's probably different to that, I can't remember of the top of my head) the value 10.1234 grams/second would be converted to 10.1234*64 = 647.8976. Since the value MUST be stored into a 16 bit integer value (with no decimal places) the value 648 would be stored in the PCM (or *.tun file). When that is read back again it would be converted back to grams/second like this: 648/64 = 10.125
The same rounding occurs with ALL calibrations, it is just more noticable in the injector flow rate calibration because it uses 4 decimal places of precision.
Plus another description here:
Quote:
It is something that we have no control over. It is not a PC or windows thing it is a GM PCM thing. It is not even really a "problem" it is just the way the PCM works.
The PCM uses a 16-bit Motorola CPU which is optimized to perform integer arithmetic on 16-bit values. The PCM stores 90% of all calibration values as 16 bit integers, not true floating point values. The other 10% are 8-bit integers.
For example:
With 16 bits you can store only 65,536 unique values. When representing a value, say Grams/Second in the range 0 to 512 (which is what the B4001 table does), then the precision is 65536/512 = 128. Each unique value of the 65536 values that can be stored is 1/128th of 512, which means that each value is quantized in steps of 0.0078125.
Precision greater than that is not possible. EFILive "rounds" all values to the nearest quantized step value.
Of interest: you can see the maximum value for grams/second is 511.992 (not 512) because 65535/128=511.992. You'll also notice that if you multiply the values displayed in the MAF table by 128, you should end up with a whole number in the range 0..65535 or very close to a whole number. Because we only use 6 decimal places, some rounding at the 5th or 6th decimal place may occur. But 0.0001 grams per second is not significant when tuning a car