G'day Rob,

Methinks I'm getting closer.

Check your file
/includes/functions/html_output.php

Specifically look for the lines that follow

// clean up the link before processing (line#72) and
// clean up the link after processing (line#91)

This is where the URL strings are cleaned up.

Also, double check that this file is the *correct* one.
There is another similar file, also called html_output.php in the /admin/includes/function/ folder.

I image it would be quite easy to edit the wrong one of these two files if not careful (much like the two configure.php files).

Although not pretty, if you find the line

return $link ; (line #96) then just befor that add

echo $link ;

You'll see all the links being dumped to screen (when you visit the site) with or without the errors.
Also, in the lines before that you'll see several places where the ampersands get manipulated/processed. If you comment out these one by one (with the echo line still in place) you will be able to quickly narrow down exactly which one of these is causing the problem... That assuming you need to continue to fault find rather than replacing the offending file.

Back in your hands now :)

Cheers
Rod