Have you made template-override changes to /includes/templates/YOUR_TEMPLATE/templates/tpl_modules_checkout_one_shipping.php?
I ask because that setting is active on my demo site and checking out with this product in the cart results in the shipping-cost block being displayed ... which isn't displayed on your test site.
Bookmarks