Search this insane blog:

Wednesday, June 2, 2010

decimal calculation Phakery

I have written a calculation inside a column called written amt.
I multiplied the Amount * Net Weight which is the Net Weight times Amount.
But when I verified the amounts by a unit test (SQL Mag’s InstantDoc ID #37428 or Wikkipedia)
certain values failed.
Notice the Net Weight column for the failed lines:

Object1: failed rows
image

All the failed rows had net weights of decimals
Cross reference with the amounts.

Then I knew the column specification for my table was smaller than the source table.
source table: decimal (38,20)
destination table (from the select): (38,2)
so, what I see are the truncated amounts shoved by the wayside to give me rounded numbers.
voila.
[insert large exclamation of “DUH” here]

No comments:

Post a Comment