Posts Tagged as ‘truncation error’

May 27, 2008

Myth: Error caused by chopping a number is called truncation error

Round off error is the error caused by approximate representation of numbers.
When people talk about round off error, it is the error between the number and its representation. For example 200/3 would be represented as 66.6667 in a six significant digit computer that rounds off the last digit. The last digit has been [...]