Fixed-Point Numbers

• is a method of representing fractional (non-integer) numbers by storing a fixed number of digits of their fractional part

Link to original

Floating-Point Numbers

• solves a number of fixed-point representation problems:
  • Fixed-Point has a fixed window of representation, which limits it from representing both very large and very small numbers
  • Fixed-Point is prone to a loss of precision when two large numbers are divided
• employs a sort of “sliding window” of precision appropriate to the scale of the number. This allows it to represent numbers from 1,000,000,000,000 to 0.0000000000000001 with ease, and while maximizing precision (the number of digits) at both ends of the scale

Link to original

Two’s Complement)

• uses the Most Significant Bit (MSB) as a sign bit to display a range of either positive numbers or negative numbers

Link to original

limited-precision decimal

Basically the same as an IEEE 754 binary floating-point, except that the exponent is interpreted as base 10. As a result, there are no unexpected rounding errors. Also, this kind of format is relatively compact and fast, but usually slower than binary formats

arbitrary-precision decimal

Sometimes called “bignum”, this is similar to a limited-precision type but has the ability to increase the length of the significand (possibly also the exponent) as required. The downside is that there is some basic overhead (memory and speed) to support this flexibility and that the longer the significand gets, the more memory is needed and the slower all calculations become.

It can be very tempting to say “My calculation is important, so I need as much precision as possible”, but in practice, the actual importance of precision at the 10,000^th decimal digit quickly pales in comparison with the performance penalty required to support it.

symbolic calculations

The “holy grail” of exact calculations. Achieved by writing a program that actually knows all the rules of math and represents data as symbols rather than imprecise, rounded numbers. For example:

• 1/3 is actually a fraction of “one divided by three”
• The square root of 2 is really the number that, multiplied by itself, is exactly 2
• Even transcendental numbers like 𝑒 and 𝜋 are known, together with their properties, so that 𝑒^𝑖𝜋 is exactly equal to -1.

However, these symbolic math systems are complex, slow, and require significant mathematical knowledge to use. They are invaluable tools for mathematicians, but not appropriate for most everyday programming tasks. And even many mathematicians work on problems that are imprecise, numerical solutions are better because no symbolic solution is known.