Hi,
If you look at the actual code contained in some of the order total modules, e.g ot_subtotal.php, ot_tax.php, ot_total.php
you will see that no actual calcualtions take place here, and all these modules do is output information that is calcualted in the order class.
If you want to change these values, this needs to be done while that calculation is done in the order class. (in 1.3.8 you should be looking at the code in the order class cart method (circa line. 420-520)
Unfortunately in the current (presently v1.3.8a) version there are no notifiers in the order class that would allow you to hook into that code, although if you want to add your own feel free.
There are other ways of tapping into the order total system, that would not require touching core code. This is how things like the discount coupons and group pricing work.
You could go down this route but it does have some complications. The new order total module would need to be installed, and 'activated' by the user. You would also need to ensure that it had the correct 'sort order' as generally this governs what other sort orders you can affect. e.g. if you wanted your order total module to affect the sub total, it would need to have a sort order that was less than that of the sub total order total module.
what would also work, and I think kuroi alluded to this, is to provide a order total module that replaced the current sub total module, you would of course need to provide instructions to your users to disable the original sub-total module.