OK,
I solved this myself.
I wondered why the last three options of the PayPal install don't show up. The debug and test alternatives.
I went looking for them in admin/tools/developer's tool kit.
I found the rows of code in paypal.php and changed the mail to Yes and put in my email address.
Then I bought the test item for around the 20th time, and voila, I got emails, just one row per email. One of them stated that port 80 is closed. Now this is funny, the ISP said it's open!
Ok, they admitted it was open only inbound, so they opened it outbound too. Now the payment works. Finally.
Summer has come.