The Lost Penny of ZC

[pilou2]

The 'lost penny' in Zen Cart refers to the one unit difference between invoice total and addition of intermediate numbers (cart total + shipping +...).
It is due to rounding errors and is not a problem specific to ZC, it is in any accounting.

When calculating total, Zen Cart uses 'high precision' calculation (no rounding before display, in ZC v2.0.0), but on invoice you have to round item prices, cart subtotal, shipping cost, discounts and tax... And they have to equal total when added, which sometimes does not work because of differences in calculation methods and more importantly in rounding time.
This problem has a high probability of being visible when there are lots of multiplications involved: High quantity of items in cart, tax calculation and the worst, currency conversions with a high rate (> 100).

There was some attempt to fix this in ZC by rounding numbers at the right place, but it was a bad idea. This method would work in certain cases but add more error in others. What was done in recent version of ZC (v2.0.0 in preparation) was to remove these rounding. Results are more consistent but still sometimes, depending on numbers, the lost penny appears (or disappears) on the invoice.
There is a case where this small error is amplified, when converting, for example, from USD to JPY at a rate of 155. Lost of few tenths of a penny when converted become few dozens. Hopefully, ZC uses at least four decimals, which is good as long as no rounding is done. Although, when multiplying by 155, one or two more decimal might be better.

The lost penny is mathematically inevitable, but it can be moved... By detecting when it appears, you can correct it. Then all numbers on the invoice will add correctly. The only drawback is that if you compare the same invoice content, but one printed with prices without tax and the other with tax included, there will be one penny difference in final total. But as we generally use only one type of invoice at a time, it will make the lost penny disappear!

I modified ot_total.php to do that. You can take a look at it on GitHub here.

[carlwhat]

...It is due to rounding errors and is not a problem specific to ZC, it is in any accounting.

The lost penny is mathematically inevitable, but it can be moved...

a philosophical discussion on accounting?

i would go as far as to state there is a structural problem in the design of zen-cart. as in, why is there an ot_total module? and why can a user change its sorting?

when one looks at an order total on an invoice, the order total should simply be the addition of all other order total modules. period. why is said module calculating anything more?

part of the problem is that the order object carries an order total ($order->info['total']) that can very easily get out of whack. and that is the essence of this problem as you have seen and are attempting to correct.

numbers on an invoice should add up. there should be no calculation more than that.

i have watched from afar your work in this arena, and applaud your effort. but accounting, at least in the US, has something called generally accepted accounting principles (GAAP), which allows for a lack of exact precision if it does not materially affect the reporting by the entity. so a rounding error, from an accounting standpoint is not really an error.

but store owners and customers want numbers on an invoice to add up. this idea of re-calculating the order total to match all of the sub-totals is a conundrum that i have wrestled with many times over the years with ZC. but a structural change to not allow changes to an order total module, and to have it simply just add up all of the order total modules is a change that i can get behind...

having something called $lost_penny_total? displaying a number on the invoice that is different than the number carried in the database?

at some point we need to get out of the weeds and look at the forest....

[pilou2]

Well. I agree with you, Zencart should just add numbers at the end. Actually, it is mixing calculation from different modules. I think it is due to all possible options and the fact it is modifying object properties in a linear algorithm way.
Adding numbers sounds easy, but which numbers when you can have taxable one, not taxable or with different tax, and what if customer wants to see it in a different currency? Modifying Zen Cart like you say would be a huge work and not as obvious as you seem to think.
I did some test in an Excel sheet with 'proper' calculation method and numbers are adding well, but there was a penny difference between gross price and net price invoices. It is what I reproduce with my quick fix, not recalculating much at all, just moving the penny difference where it should be. Still, as you said, I don't like to display a number and save another one in database...

About the fact that customer and store owners want numbers on an invoice to add is in my opinion perfectly normal. The invoice is what defines the transaction between the two, the accounting use comes after. What is an accountant acceptable rounding might just be an unacceptable error for a customer. Even in accounting now, when dealing with computer transactions, as far as I know, tolerance is pretty low. For cash transaction it is another story, there are probably as many methods as administration when dealing with rounding. I think I have read somewhere that Canada has stopped producing one penny coin, so cash transaction in computer systems would be easier to manage...

Finally, I am worried by the fact that a store owner changing order of 'ot' modules would completely mess up tax and total calculations...

[carlwhat]

...Still, as you said, I don't like to display a number and save another one in database...

i think this gets to be a problem.

what happens to the number sent to the payment provider? ie paypal, square or authorize.net... the invoice will say 1 number and the amount collected says another.

for those owners that keep track of payments, there is an inherent discrepancy.

i am not against what you are trying to do, ie have all of the numbers on the invoice add up; but that should be the number stored in the database. and in a cursory look at your code, it is not.

best.

[pilou2]

...
i am not against what you are trying to do, ie have all of the numbers on the invoice add up; but that should be the number stored in the database. and in a cursory look at your code, it is not.

best.

It is easy to save the corrected total in database. I already modified code on GitHub. Like this, everything is consistent.
Don't worry, I appreciate your opinion and I don't mind you being against what I propose as long as you tell me why. ;)

This penny correction looks simple and quickly made, but I actually spend a lot of time (and more...) on the discount coupon module recently, which has influence that modification. I did try other things that did not give good results or were too complicated.

[mc12345678]

Adding input ideally only to philosophy: order total I see as related to the transaction and cost to customer, the entirety of the ot_XXXXX "utilities" offers operations against previously calculated (by whatever means) values such that operations can occur to the store after determining the customer's cost: e.g. non priced "awards" not yet available for spending, discounts or additional fees. The position of that feature being dependent on its basis for determination and impact.

I see two topics in discussion though perhaps neither do directly stated: one size to fit all (code completely address all things for all conditions even when all are in one transaction) and user error in making adjustments. Neither of these can be immediately fully resolved. I have seen that homework was being done to attempt to identify all (most) reasonable configurations; however, even within each proposed condition localities and nations have yet more complexities.

As to user error, there is instruction (at many places and by many ways) or as proposed more of a lock down to force the software to work the "Zen Cart" way which I perceive to do no more than identify the need to find something else with less imposition. A psychological affect may be to encourage more discussion about how else it should be done, but at what cost?

As to JPY, I've previously looked at how that currency is handled and as memory serves, forcing to a unruly value sends to happen too early in the transaction calculations and almost too abruptly. My not knowing associated currency/business requirements prevented me from making any useful suggestions.

The penny issue unfortunately does happen also with cash transactions though not always as evident to the shopper as when looking at a breakdown on screen or shortly after ordering. An example I have was that once I was in line at a vendor that sold one product and that could be purchased in multiple quantity. While in line I saw someone buy a quantity of one followed by someone purchasing a quantity of two. The per item cost was less in a quantity of two because of tax and associated rounding. I had considered purchasing the second later but as a customer it would cost me more, even if only slightly.

Lastly in mind at the moment, I realize it appears the discussion is centered around the invoice (a document made available after making the purchase) it seems the same philosophical process to get to the same result could be applied earlier in the purchase so that the customer has consistent information through the entire process. As to list change, it seems to me that the customer "experience" should be paramount and that whether the owner has to internally adjust to be in compliance on a per purchase or some periodic basis that it is a cost of doing business even if those same owners are customers (not directly paying) to Zen Cart.

[pilou2]

Well, to come back to code, I updated it so it goes through all modules used during checkout and add their results, rounded as displayed. It is then compared with actual order total which is updated by each module by adding their results too but one after another and in full precision.
To do that, I had to add a line in each optional modules to save their own result in order object in a table named 'option_modules' (actually "$order->info['option_modules']").
I did some orders with all possible modules used and results were correct, penny correction worked fine too. For these tests I used modified versions of coupons and GV modules (See branch Coupons on GitHub) as originals have some bugs.

Order of module is very important as they use precedent module order total to do their calculations. Default setting should probably be modified, I would place tax module at order 998 just before total module (999).