QP School
Getting accurate results when working with integers in Ruby - Printable Version

+- QP School (https://qomplainerzschool.lima-city.de)
+-- Forum: Tutorials (https://qomplainerzschool.lima-city.de/forumdisplay.php?fid=3)
+--- Forum: Ruby 3 Tutorials, exercises and examples (https://qomplainerzschool.lima-city.de/forumdisplay.php?fid=17)
+--- Thread: Getting accurate results when working with integers in Ruby (/showthread.php?tid=2498)



Getting accurate results when working with integers in Ruby - Qomplainerz - 02-27-2021

After reading the official Ruby 3.0.0 documentation I have discovered that there are 2 ways to get the square root of a number.
When working with small numbers, both options return the same results,
but when the numbers get bigger the results of Option 2 may be inaccurate as follows:

Option 1:


Quote:Num1 = Integer.sqrt(10**46)
puts Num1


This will put exactly 100000000000000000000000 on the screen.

Option 2:


Quote:Num1 = Math.sqrt(10**46).floor
puts Num1


This will put 99999999999999991611392 on the screen.