1. I mention gzip because right after the language text is displayed, there is a PHP flush() command which dumps out any cached browser contents to be displayed. If your server and your site have a clash on gzip settings, there's a slight chance that some stray characters might display (although the chance is almost nil).
Admin->Configuration->GZIP Compression ... turn it off if you have it on. If it makes no difference, then that's not the issue.

2. Is it possible that the language files have corruption in them even though your text editor doesn't show it? Perhaps refreshing them on your server from originals might be a start?

3. What do you see in View Source when the garbage characters appear?