The problem is not with zen_output_string_protected.

Your problem is that you're using a payment module written for an old version of Zen Cart, and it's passing very long error messages via the URL instead of via the $messageStack.

Rewrite your module to use current methods and you'll be fine.