round-off error
noun1
The difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic.
Numerical analysis
noun
The difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic.
Numerical analysis
Entry derived from the Wiktionary, under licence CC BY-SA 4.0 — list of authors.
Do you accept non-essential analytics cookies?