Comment on IEEE 754
cyborganism@lemmy.ca 2 weeks agoSo you have to remember how it’s represented in the system with how the bits are used? Or you just have to remember some general rules that “if you do that, it’ll fuck up.”
Comment on IEEE 754
cyborganism@lemmy.ca 2 weeks agoSo you have to remember how it’s represented in the system with how the bits are used? Or you just have to remember some general rules that “if you do that, it’ll fuck up.”
scratchee@feddit.uk 2 weeks ago
Well, rules like “all integers can be represented up to 2^24” and “10^-38 is where denormalisation happens” are helpful, but I often have to figure out why I got a dodgy value from first principles, floating point is too complicated to solve every problem with 3 general rules.
I wrote a float from string function once which obviously requires the details (intentionally low quality and limited but faster variant since the standard version was way too slow).