Paypal IPN Failing suddenly

I had zencart 1.3.8 running and I started getting daily paypal IPN warnings and paypal just listing all paypal IPN requests as retrying and then eventually failing. I have express and pro payment modules updated, all settings correct, ipn url correct on paypals side.

https://www.idosersoftware.com/extras/ipncheck.php

Comes back OK. Lunarpages added all paypal IPs to firewall, etc. Still failing.

MySQL 5.1.63-cll
Apache/2.0.64 (Unix) mod_ssl/2.0.64 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
PHP Version: 5.2.17 (Zend: 2.2.0)
Server OS: Linux 2.6.18-194.26.1.el5

chriscana:

I had zencart 1.3.8 running and I started getting daily paypal IPN warnings and paypal just listing all paypal IPN requests as retrying and then eventually failing.

Exactly what were the warnings?

chriscana:

I have express and pro payment modules updated, all settings correct, ipn url correct on paypals side.

Why are you even using PayPal IPN when you have express and pro installed?

Oh, and you really MUST upgrade your zencart installation. If you are running V1.3.8 as you state you will be hacked (assuming that this isn't your problem already).

Cheers
Rod

Sorry - meant to add that I HAD 1.3x running and I have since upgraded to 1.5, new install and database. The warning email I get is from paypal and is basically just a generic warning email that says IPNs are failing. If I look in my paypal profile all IPNs are listed as RETRYING status. The problem started with 1.3.8 and thats a reason i went to 1.5 to see if that was the issue.

I have the express module running (for paypal payment option) and Payments Pro (USA) also for taking CC payments directly on site. I do not have Stardard - IPN running. The express paypal option requires the addition of an IPN address, which I have entered correctly on paypals side.

My hosting provider has done everything from adding paypals IPs to the firewall to running tests with the firewall open. Any support email to paypal just comes back with some generic email about what IPs to add to a firewall and generic troubleshooting tips.

I have read through the IPN help article here. All seems like it should be a go but still daily warning emails from paypal. Eventually after a few days IPns get disabled on paypals side, but it does let me re-enable them.

RodG has forgotten that PayPal sends IPN notifications even on Express Checkout (and other) transactions. Zen Cart is smart enough to know which ones to ignore, and doesn't use them for adding new transactions to the store, unlike the old "paypal ipn payment module", to which RodG was referring.

Some things to check:

• did you update the /ipn_main_handler.php file when you did your site upgrade?
• did you do a thorough site cleanup before upgrading from 1.3.8, to be absolutely 100% certain you hadn't been hacked? http://www.zen-cart.com/wiki/index.php/Recovering_From_Hacks If you didn't, then you should. Never hurts to do it regularly.
• have you missed any of these troubleshooting steps? http://www.zen-cart.com/wiki/index.php/PayPal
• are you using http or https in the URL you set in your PayPal IPN settings? Have you tried changing it to the other?
• what are your PayPal ipn_*.log debug logs showing? Enable the debug-log-to-file option in your EC module. http://www.zen-cart.com/showthread.php?61199-Debug-Logging-PayPal
• What addons do you have on your site?

I did not upgrade 1.3x - I did a full new install of 1.5 with a new data base, so ipn_main_handler is at 1.5 version
IPN is using HTTPS (as my site is https) but I can try changing to http.
I did run through all troubleshooting.
I did logs for a day but I don't know what im looking at.
Rar'd log dir is here is someone would kindly take a quick look:
link removed

Upon cursory investigation, it looks like your SSL cert is registered to your domain name without a www. prefix, yet your store and your paypal configuration are both using the www. prefix.
And you're getting SSL certificate errors on all your secure pages because of insecure assets being displayed.

Hm. Ok. Im no SSL expert, but what should my settings be and where to correct this?

It would be best if your store used the same URL as your SSL certificate is registered for. Best if PayPal URL uses the same, or try using http instead.
As for the insecure assets issue: http://www.zen-cart.com/content.php?185-this-page-contains-both-secure-and-nonsecure-items-%28or-unauthenticated-content-or-connection-partially-encrypted%29

Neither of these is a magic fix. Just something noticed while looking at your site.

A few followups:

Where do I change my store url to match my ssl url and how do I know what my ssl url is?
With the unsecure assets - would easypage links to external http addresses cause this, or only things like http banners?
I will try changing my paypal ipn to http instead of https and see if there is any resolution.

DrByte said the certificate is registered to http://yourdomain.com not http://www.yourdomain.com
You would have entered the URL when registering your certificate, and your hosting company should be able to confirm this.

Your domain is referenced in a few places: both configure files and Paypal's site (the return URL) are the most obvious in relation to your problem.

As far as I know external links (just a simple text a href link) would not trigger an SSL error, it's embedded content (images, javascript, iframes etc)

I checked my configure.php files and server configs are set to https without the www. It was like this, so I think it is OK on that side. I have tried both http and https on the paypal side for configuring the IPN (without the www), but still getting fails. My hosting provider has run all kinds of firewall tests and they are now saying that maybe paypal is blocking on their firewall. Any support request to paypal gets a generic IPN troubleshooting email.

The response error on paypals side is "retrying" which they have listed as: Retrying - indicates that PayPal did not receive a 200ok and the message was resent between 1 and 15 times and that PayPal is continuing to resend the message.

The only ipn_xxxxx.log files you supplied are those that come from people using browsers to access the ipn_main_handler.php file. There's no evidence of PayPal's servers actually reaching that file. You'll have to work with PayPal to find out what the actual response code was. Their generic answer is that it didn't get a 200 response. But surely you can get them to tell you what response they are getting. You'll have to escalate your MTS ticket to do that.

Maybe you have /.htaccess rules set up that are interfering with normal operation. There are many URL-rewriter addons that attempt to change the way Zen Cart writes its URLs, and if implemented improperly then the symptoms you describe are not surprising.)
(Zen Cart only supplies .htaccess files for subfolders. It does NOT come with any .htaccess in the "root" of your store, so any .htaccess file in the same folder as your ipn_main_handler.php file is something you added yourself. And the rules you've got in there could be interfering.)

I don't have any url rewriter add-ons and the .htaccess files are only the ones put there by a default zen install. None in either the root or the same folder as the ipn_main_handler. I have tried opening several tickets with paypal and it seems to matter how I word it mts just sends me a generic troubleshooting doc and IP address list with little other information and then closes the ticket.

Can you give me those ticket numbers?

Incident: 120628-000140
Incident: 120722-000025

Issue was happening (to start) on 1.3x install after working fine for a long time. Took the opportunity to do a new install with 1.5, new database, everything in efforts to support the ipn issue.

My own testing by asking the PayPal simulator to send a test IPN to you is also getting instantly rejected. I can't tell what the actual response is, but hopefully the MTS folks can dig into it further. I've asked for an escalation to see if something further can be explained from PayPal's end.

However, it does seem to me as though something in your server is specifically blocking PayPal's servers from accessing. I can get to your site from a browser without difficulty, so it seems that your host's firewall is being choosy.

If they can't sort it out, you do have the option of turning off the IPN feature in your PayPal account's preferences. The downside to that is that your store will not be notified of refunds or echecks clearing or anything else that might affect existing transactions. It won't prevent Express Checkout or Website Payments Pro from working properly. It just prevents "updates" from being passed on to your store automatically.

thank you. I also have had an existing ticket open with Lunarpages for awhile now and they have tried adding all paypal IPs to the firewall all the way to disabling it, but its still open and hopefully they are still troubleshooting it also.

As it stands, I get daily emails from paypal and eventually it disables itself. I am able to re-enable it, but hopefully paypal or lunarpages gives some idea.

Two additional replies to IPN support requests from Paypal and Lunarpages. Lunarpages generically blames paypal and paypal does the same and blames lunarpages. Nobody gives me any additional technical info other than "we don't know" and I'm back to square one.

Do I need to give up on IPNs?

I just got word back from one of my contacts about this. Apparently PayPal has been suffering from an IPN bug that was causing these problems for you.

a scheduled fix should be pushed out sometime tomorrow. The IPN messages that were affected will not be resent, to avoid any confusion for merchants that created a workaround for the time-being.

So, if your IPN switch gets disabled again before Wednesday, turn it back on on Wednesday and theoretically things should be better.

If the problem persists after that, re-open one of those tickets or log a new one (quoting those last 2 incident numbers) and post the new incident number here again.

I really appreciate you looking into this. I will wait until the end of the week and re-enable IPN is it gets auto-disabled again.