Quantcast
Channel: SQLServerCentral » SQL Server 2014 » Development - SQL Server 2014 » Latest topics
Viewing all articles
Browse latest Browse all 3145

Strange behavior with decimals

$
0
0
Run the following sql:DECLARE @results TABLE(r INT, d DECIMAL(38,37))DECLARE @X DECIMAL(3,2) = 1.00INSERT INTO @results(r,d) VALUES(0, @X/7)INSERT INTO @results(r,d) VALUES(1, @X/7.1);INSERT INTO @results(r,d) VALUES(2, @X/7.01);INSERT INTO @results(r,d) VALUES(3, @X/7.001);INSERT INTO @results(r,d) VALUES(4, @X/7.0001);INSERT INTO @results(r,d) VALUES(5, @X/7.00001);INSERT INTO @results(r,d) VALUES(6, @X/7.000001);INSERT INTO @results(r,d) VALUES(7, @X/7.0000001);INSERT INTO @results(r,d) VALUES(8, @X/7.00000001);INSERT INTO @results(r,d) VALUES(9, @X/7.000000001);INSERT INTO @results(r,d) VALUES(10, @X/7.0000000001);INSERT INTO @results(r,d) VALUES(11, @X/7.00000000001);INSERT INTO @results(r,d) VALUES(12, @X/7.000000000001);INSERT INTO @results(r,d) VALUES(13, @X/7.0000000000001);INSERT INTO @results(r,d) VALUES(14, @X/7.00000000000001);INSERT INTO @results(r,d) VALUES(15, @X/7.000000000000001);INSERT INTO @results(r,d) VALUES(16, @X/7.0000000000000001);INSERT INTO @results(r,d) VALUES(17, @X/7.00000000000000001);INSERT INTO @results(r,d) VALUES(18, @X/7.000000000000000001);INSERT INTO @results(r,d) VALUES(19, @X/7.0000000000000000001);INSERT INTO @results(r,d) VALUES(20, @X/7.00000000000000000001);INSERT INTO @results(r,d) VALUES(21, @X/7.000000000000000000001);INSERT INTO @results(r,d) VALUES(22, @X/7.0000000000000000000001);INSERT INTO @results(r,d) VALUES(23, @X/7.00000000000000000000001);INSERT INTO @results(r,d) VALUES(24, @X/7.000000000000000000000001);INSERT INTO @results(r,d) VALUES(25, @X/7.0000000000000000000000001);INSERT INTO @results(r,d) VALUES(26, @X/7.00000000000000000000000001);INSERT INTO @results(r,d) VALUES(27, @X/7.000000000000000000000000001);INSERT INTO @results(r,d) VALUES(28, @X/7.0000000000000000000000000001);INSERT INTO @results(r,d) VALUES(29, @X/7.00000000000000000000000000001);INSERT INTO @results(r,d) VALUES(30, @X/7.000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(31, @X/7.0000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(32, @X/7.00000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(33, @X/7.000000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(34, @X/7.0000000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(35, @X/7.00000000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(36, @X/7.000000000000000000000000000000000001);INSERT INTO @results(r,d) VALUES(37, @X/7.0000000000000000000000000000000000001);The results are:r...........d0...........0.14428500000000000000000000000000000001...........0.14225300000000000000000000000000000002...........0.14407900000000000000000000000000000003...........0.14426510000000000000000000000000000004...........0.14428365000000000000000000000000000005...........0.14428550800000000000000000000000000006...........0.14428569360000000000000000000000000007...........0.14428571222000000000000000000000000008...........0.14428571407900000000000000000000000009...........0.144285714265100000000000000000000000010..........0.144285714283650000000000000000000000011..........0.144285714285508000000000000000000000012..........0.144285714285693600000000000000000000013..........0.144285714285712220000000000000000000014..........0.144285714285714079000000000000000000015..........0.144285714285714265100000000000000000016..........0.144285714285714283650000000000000000017..........0.144285714285714285500000000000000000018..........0.144285714285714285600000000000000000019..........0.144285714285714285000000000000000000020..........0.144285714285714280000000000000000000021..........0.144285714285714200000000000000000000022..........0.144285714285714000000000000000000000023..........0.144285714285710000000000000000000000024..........0.144285714285700000000000000000000000025..........0.144285714285000000000000000000000000026..........0.144285714280000000000000000000000000027..........0.144285714200000000000000000000000000028..........0.144285714000000000000000000000000000029..........0.144285710000000000000000000000000000030..........0.144285700000000000000000000000000000031..........0.144285000000000000000000000000000000032..........0.144285000000000000000000000000000000033..........0.144285000000000000000000000000000000034..........0.144285000000000000000000000000000000035..........0.144285000000000000000000000000000000036..........0.144285000000000000000000000000000000037..........0.1442850000000000000000000000000000000Notice how the precision for the numbers used in the inserts is always increasing, but in the results the precision increases up to 20 in rows 16 and 17 and then goes down to 6.Changing the precision of @X changes the point at which the precision in the results starts to decrease, but the pattern is always the same: it either starts at the max precision or it builds up to that max and then it goes down to 6 decimal places.Why is that?

Viewing all articles
Browse latest Browse all 3145

Trending Articles