Getting accurate results when working with integers in Ruby - Printable Version +- QP School (https://qomplainerzschool.lima-city.de) +-- Forum: Tutorials (https://qomplainerzschool.lima-city.de/forumdisplay.php?fid=3) +--- Forum: Ruby 3 Tutorials, exercises and examples (https://qomplainerzschool.lima-city.de/forumdisplay.php?fid=17) +--- Thread: Getting accurate results when working with integers in Ruby (/showthread.php?tid=2498) |
Getting accurate results when working with integers in Ruby - Qomplainerz - 02-27-2021 After reading the official Ruby 3.0.0 documentation I have discovered that there are 2 ways to get the square root of a number. When working with small numbers, both options return the same results, but when the numbers get bigger the results of Option 2 may be inaccurate as follows: Option 1: Quote:Num1 = Integer.sqrt(10**46) This will put exactly 100000000000000000000000 on the screen. Option 2: Quote:Num1 = Math.sqrt(10**46).floor This will put 99999999999999991611392 on the screen. |