Re: percentage discount not calculating properly

[Luke <..hidden..>]

On Tue, 23 Feb 2010, Bob Miller wrote:

See David's message for a supposed fix, but as for what's going on...:

> One of my clients has recently noticed a slight peculiarity in the
> percentage discount, and I have been able to verify it.
> For example, sell 252 items at $1.05 with a 15% discount.  By my handy
> dandy calculator application, 252*1.05*.85=224.91.  However, on the
> invoice, the total calculated is $224.28.  It is a small difference,
> however, it grows when more items are sold.  When we put on another

Your math is not the math which the program does.  It does
this:
1.05 * 0.85 * 252.  That would still be okay, until you show your work.

1.05 * 0.85 = 0.8925
0.8925 Internal round to 2 decimal places = 0.89
0.89 * 252 = 224.28

This has bugged me to no end.  SL had a way to change the precision, but 
LSMB dropped it for reasons I knew not, and provided no workaround for 
this problem [until now].

 > invoice 784 of the same item being sold at $1.05 with the same 
> 15% discount, 784*1.05*.85=699.72, however the invoice states under the
> extended column that the value should be $697.76, which is a discrepancy
> now measuring in full dollars.

784 * (round)(1.05 * .85) = 697.76

> Conversely, selling 7 items at $100 with the same 15% discount, 7*
> $100*.85=$595, and it calculates this properly.  But, if I sell those

That was a clue to what was happening.:)

100 * .85 = 85.00
85.00 * 7 = 595

> items at $100.05, it takes things out of whack again:  7*$100.05*.85=
> $595.2975($595.30), however the invoice calculates $595.28.  Again, this
> is only 2 cents, but that is obviously going to be a problem over a
> period of time.

100.05 * .85 = 85.0425
85.0425 (round 2) = 85.04
85.04 * 7 = 595.28

It looks at the quantity last, whereas we think of discounting after 
considering the quantity.
That behavior, at least, is actually correct, for percentage based 
discounting.
What is not correct is the rounding.

> My first suspicion is that this is a rounding issue, but the difference
> doesn't work out to be a single penny per item sold.  It also occurred
> to me that it may be applying the discount before or after something
> else is being calculated on that number, such as tax or another
> discount, but again I don't make the numbers add up to anything that
> supports that theory.
> Anybody have any theories on what might be causing this?  Or one better,
> how to fix it?  Any comments are much appreciated...

Regards,

Luke