I changed the email address on the customer account and it went though to the checkout successful page this time. I did get an email with the following though so something isn't going right...
A message that you sent contained one or more recipient addresses that were incorrectly constructed:

<>: missing or malformed local part

This address has been ignored. There were no other addresses in your message, and so no attempt at delivery was possible.