You would need to customize the module, I am guessing you are using the Modules ... Order Totals ... Shipping ot_shipping ... for this setting, and put in an extra test to see if any of the products_id that are not allowed the Free Shipping are in the cart and then prevent this from executing ...
There are also places in checkout and shipping estimator for this as well ... a search on:
MODULE_ORDER_TOTAL_SHIPPING_FREE_SHIPPING_OVER
will show you where ...
Another method is to use the Free Shipping Options freeoptions and do the test on that shipping method ...
The advantage to using the Shipping module is that you can customize it to run at the same time as your other shipping modules so that the customer has a choice to take the Free Shipping or to use another method of shipping if they are in a hurry etc. ...
Plus, using the shipping module for the Free Shipping Options freeoptions has only one file that needs customizing rather than several ...