PayPal IPN/PDT looks good, but no products or emails

[robax]

This is a permanent problem with ZC on my server. I posted endlessly about it a year and a half ago to no result, but here it is again now that I've given it an extra week of late nights because of the 139a update.

The summary:

PayPal successfully processes payments by card and by PayPal account. ZC receives full payment details of the orders, including all IPN, PDT and status info, but no products are listed in my orders and no emails are sent out. No errors are logged or displayed.

The orders table shows all the right information, but nothing is recorded in the orders_products table.

Technical details:

Cart version: Fresh install of ZC 138a or 139a, or any of my live carts - It makes no difference.
No mods or addons in the fresh installs - makes no difference.
Template also makes no difference.

Server2008 x64 dedicated server at Server Intellect.
php-5.2.13-nts-Win32-VC6-x86.msi in FastCGI mode
mysql-essential-5.1.46-winx64.msi
Previous versions of these tools behaved the same.


Here are some test results:

PayPal IPN with and without PDT communcations appear to work without errors.

IPNCHECK.PHP - Version 1.3.9:
IPN TESTING - Response Received via CURL -- COMMUNICATIONS OKAY
IPN TESTING - Confirmation/Validation response with fsockopen(): FSOCKOPEN() RESPONSE RECEIVED - Communications OKAY


CURL Test Page:
Data validation: Good
Array
(
[url] => "http://www.zen-cart.com/testcurl.php"
[content_type] => text/html
[http_code] => 200
[header_size] => 275
[request_size] => 212
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0.218
[namelookup_time] => 0.156
[connect_time] => 0.187
[pretransfer_time] => 0.187
[size_upload] => 37
[size_download] => 2183
[speed_download] => 10013
[speed_upload] => 169
[download_content_length] => 2183
[upload_content_length] => 0
[starttransfer_time] => 0.218
[redirect_time] => 0
)




Here's a typical order (with certain content obscured):


IPN PROCESSING INITIATED.
*** Originating IP: 66.211.170.66 notify.paypal.com


Breakpoint: 0 - Initializing debugging.


Breakpoint: Flag Status:
isECtransaction = 0
isDPtransaction = 0


IPN INFO - POST VARS received (sorted):
Array
(
[address_city] => MyTown
[address_country] => MyCountry
[address_country_code] => NZ
[address_name] => Harry Potter
[address_state] => MyTown
[address_status] => unconfirmed
[address_street] => 14/B MyStreet st
[address_zip] => 6011
[business] => [email protected]
[charset] => windows-1252
[custom] => zenid=0q9t4c9f1g6n6iu63c32onmco85
[first_name] => Harry
[handling_amount] => 0.00
[item_name] => Rob's test store Purchase
[item_number] => Store Receipt
[last_name] => Potter
[mc_currency] => NZD
[mc_fee] => 0.48
[mc_gross] => 1.00
[notify_version] => 2.9
[payer_email] => hairypothead##########################
[payer_id] => 5AH6XFR4DH3J9W
[payer_status] => unverified
[payment_date] => 21:46:31 Apr 29, 2010 PDT
[payment_fee] =>
[payment_gross] =>
[payment_status] => Completed
[payment_type] => instant
[protection_eligibility] => Ineligible
[quantity] => 1
[receipt_id] => 2194-8294-8803-3006
[receiver_email] => [email protected]
[receiver_id] => GYJLWUGALKL5S
[residence_country] => NZ
[shipping] => 0.00
[tax] => 0.00
[transaction_subject] => zenid=0q9t4c9f1g6n6iu63cvonmco85
[txn_id] => 7JL69866XM426860F
[txn_type] => web_accept
[verify_sign] =>
Ai1PaghZhxdfg5FmBLCDCTQpwG8jB264AbpBgxdfgxdfgxdhVXZMtQcU1rVr3
)


IPN INFO - POST VARS to be sent back (unsorted) for validation (using CURL):

To: www.paypal.com:80
Array
(
[cmd] => _notify-validate
[mc_gross] => 1.00
[protection_eligibility] => Ineligible
[address_status] => unconfirmed
[payer_id] => 5AH6XFRDH3J9W
[tax] => 0.00
[address_street] => 14/B MyStreet st
[payment_date] => 21:46:31 Apr 29, 2010 PDT
[payment_status] => Completed
[charset] => windows-1252
[address_zip] => 6011
[first_name] => Harry
[mc_fee] => 0.48
[address_country_code] => NZ
[address_name] => Harry Potter
[notify_version] => 2.9
[custom] => zenid=0q9t4c9f1g6n6iu63cvonmco85
[payer_status] => unverified
[business] => [email protected]
[address_country] => MyCountry
[address_city] => MyTown
[quantity] => 1
[verify_sign] =>
Ai1PaghZhxdfg5FmBLCDCTQpwG8jB264AbpBgxdfgxdfgxdhVXZMtQcU1rVr3
[payer_email] => hairypothead##########################
[txn_id] => 7JL69866XM426860F
[payment_type] => instant
[last_name] => Potter
[address_state] => MyTown
[receiver_email] => [email protected]
[payment_fee] =>
[receiver_id] => GYJLWUGALKL5S
[txn_type] => web_accept
[item_name] => Rob's test store Purchase
[mc_currency] => NZD
[item_number] => Store Receipt
[residence_country] => NZ
[receipt_id] => 2194-8294-8803-3006
[handling_amount] => 0.00
[transaction_subject] => zenid=0q9t4c9f1g6n6iu63cvonmco85
[payment_gross] =>
[shipping] => 0.00
)


CURL OPTS: Array
(
[10002] => https://www.paypal.com/cgi-bin/webscr
[47] => 1
[10015] =>
cmd=_notify-validate&mc_gross=1.00&protection_eligibility=Ineligible&address_status=unconfir med&payer_id=5AH6XFRDH3J9W&tax=0.00&address_street=14%2FB+MyStreet+st&payment_da te=21%3A46%3A31+Apr+29%2C+2010+PDT&payment_status=Completed&charset=windows-1252&address_zip=6011&first_name=Harry&mc_fee=0.48&address_country_code=NZ&addre ss_name=Harry+Potter&notify_version=2.9&custom=zenid%3D0q9t4c9f1g6n6iu63cvonmco8 5&payer_status=unverified&business=sales%40mydomain.com&address_country=New+Zeal and&address_city=MyTown&quantity=1&verify_sign=Ai1PaghZh5FmBLCDCTQpwG8jB264AbpBN M5hx8MsiDxVXZMtQcU1rVr3&payer_email=hairypothead%40hotmail.com&txn_id=7JL69866XM 426860F&payment_type=instant&last_name=Potter&address_state=MyTown&receiver_emai l=sales%40mydomain.com&payment_fee=&receiver_id=GYJLWUGALKL5S&txn_type=web_accep t&item_name=Rob%27s+test+store+Purchase&mc_currency=NZD&item_number=Store+Receip t&residence_country=NZ&receipt_id=2194-8294-8803-3006&handling_amount=0.00&transaction_subject=zenid%
D0q9t4c9f1g6n6iu63cvonmco85&payment_gross=&shipping=0.00
[13] => 45
[78] => 30
[41] => 0
[42] => 0
[52] => 0
[19913] => 1
[64] => 0
[81] => 2
[75] => 1
[74] => 1
[84] => 2
[10018] => Zen Cart(tm) - IPN Postback
)


CURL response: VERIFIED


IPN INFO (cl) - Confirmation/Validation response VERIFIED


Breakpoint: 1 - Collected data from PayPal notification


IPN INFO :: Transaction email details.
From IPN = [email protected] | [email protected] From CONFIG = [email protected]


Breakpoint: 2 - Validated transaction components


Breakpoint: 3 - Communication method verified


IPN NOTICE :: Could not find matched txn_id record in DB. Therefore is new to us.


Breakpoint: 4 - Details: txn_type=unique ordersID = 0 IPN_id=0
Relevant data from POST:
txn_type = unique
parent_txn_id = None
txn_id = 7JL69866XM426860F


Breakpoint: 5 - Transaction type (txn_type) = unique [parentLookup=unique]


IPN INFO :: Currency/Amount Details:
PayPal email address = [email protected] | mc_currency = NZD | submitted_currency = NZD | order_currency = NZD | mc_gross = 1.00 | converted_amount = 1.00 | order_amount = 1.0000125


Breakpoint: 5a - built order -- OID: 22


Breakpoint: 5b - PP table OID: Array
(
[order_id] => 22
[txn_type] => unique
[module_name] => paypal (ipn-handler)
[module_mode] => IPN
[reason_code] =>
[payment_type] => instant
[payment_status] => Completed
[pending_reason] =>
[invoice] =>
[mc_currency] => NZD
[first_name] => Harry
[last_name] => Potter
[payer_business_name] =>
[address_name] => Harry Potter
[address_street] => 14/B MyStreet st
[address_city] => MyTown
[address_state] => MyTown
[address_zip] => 6011
[address_country] => MyCountry
[address_status] => unconfirmed
[payer_email] => hairypothead##########################
[payer_id] => 5AH6XFRDH3J9W
[payer_status] => unverified
[payment_date] => 2010-04-29 21:46:31
[business] => [email protected]
[receiver_email] => [email protected]
[receiver_id] => GYJLWUGALKL5S
[txn_id] => 7JL69866XM426860F
[parent_txn_id] =>
[num_cart_items] => 1
[mc_gross] => 1.00
[mc_fee] => 0.48
[settle_amount] => 0
[settle_currency] =>
[exchange_rate] => 1
[notify_version] => 2.9
[verify_sign] =>
Ai1PaghZhxdfg5FmBLCDCTQpwG8jB264AbpBgxdfgxdfgxdhVXZMtQcU1rVr3
[date_added] => now()
[memo] => {Record generated by IPN} [ProtectionEligibility:Ineligible]
)


Breakpoint: 5c - PP table OID saved


Breakpoint: 5d - PP hist ID: 12


Breakpoint: 5e - PP hist_data:Array
(
[paypal_ipn_id] => 12
[txn_id] => 7JL69866XM426860F
[parent_txn_id] =>
[payment_status] => Completed
[pending_reason] =>
[date_added] => now()
)


Breakpoint: 5f - PP hist saved


Breakpoint: 5g - new status code: 2


Breakpoint: 5j - order stat hist update:Array (
[orders_id] => 22
[orders_status_id] => 2
[date_added] => now()
[comments] => **** ADDRESS OVERRIDE ALERT!!! **** CHECK PAYPAL ORDER DETAILS FOR ACTUAL ADDRESS SELECTED BY CUSTOMER!!
PayPal status: Completed @ 21:46:31 Apr 29, 2010 PDT Trans ID:7JL69866XM426860F
Amount: 1.00 NZD
[customer_notified] =>
)


I've read hundreds of posts over and over again, but they tend to be about communications problems, where I'm not having any, as far as I can tell.

I have been through all points on the PayPal troubleshooting page several times... ports, ip's and all.

I have tried it with and without PDT, but it doesnt matter either way.

I'm not using the Sandbox - I'm testing with live payments.

I've removed the previous versions of mysql and PHP and then installed the latest releases. I did everything again from scratch after removing all traces of the previous tools. All ini files, handlers, permissions accounts etc are new again this time, with the same result.

There is one clue I can provide:

In my endless tests, I have had a small number of orders show up with products and notifications all good.
I have examined these orders closely and they are different from the bad ones in two ways:

Successful orders say "web_accept" in the "Transaction Type" column.
Unsuccessful orders say "unique" in the same column.
And:
Successful orders record the ip address as two identical ips" "118.92.xxx.xxx - 118.92.xxx.xxx"
Unsuccessful orders record it as two different ips: "118.92.xxx.xxx - 66.211.170.66"
I know that the 118 ip is my ISP connection, whist the 66 ip is PayPal.

Both successful and unsuccessful orders display full IPN data with payer ID, Transaction ID, status completed and everything else looking good.

But no products listed and no notifications going out??

I run 3 other types of shopping cart (ASP) on the same server and they all use PayPal IPN and they all work without trouble.

So what is wrong with ZenCart? The payments are being processed... Why can't it just show me the products and send out the emails?

Any ideas are much appreciated.
Rob

[DrByte]

The summary:

PayPal successfully processes payments by card and by PayPal account. ZC receives full payment details of the orders, including all IPN, PDT and status info, but no products are listed in my orders and no emails are sent out. No errors are logged or displayed.


I have tried it with and without PDT, but it doesnt matter either way.

I'm not using the Sandbox - I'm testing with live payments.

Most puzzling.
In the endless testing I've done with this, I've never once encountered any such symptoms.

There is one clue I can provide:

In my endless tests, I have had a small number of orders show up with products and notifications all good.
I have examined these orders closely and they are different from the bad ones in two ways:

Successful orders say "web_accept" in the "Transaction Type" column.
Unsuccessful orders say "unique" in the same column.
And:
Successful orders record the ip address as two identical ips" "118.92.xxx.xxx - 118.92.xxx.xxx"
Unsuccessful orders record it as two different ips: "118.92.xxx.xxx - 66.211.170.66"
I know that the 118 ip is my ISP connection, whist the 66 ip is PayPal.

The orders that show two identical IPs are those that completed via PDT. The ones with PayPal's IP as the 2nd address were completed without PDT.

The orders table shows all the right information, but nothing is recorded in the orders_products table.

...

But no products listed and no notifications going out??

What's interesting about those two statements is that the order header (in the orders table) is written properly but then it's as if the rest of the code execution is halting on you.

Are you finding any myDebug-xxxxxx.log files in your /cache/ folder ? If certain PHP or MySQL errors are occurring, I would expect an error to be logged if the product data couldn't be inserted for a particular reason.

[robax]

Most puzzling.

What's interesting about those two statements is that the order header (in the orders table) is written properly but then it's as if the rest of the code execution is halting on you.

Are you finding any myDebug-xxxxxx.log files in your /cache/ folder ? If certain PHP or MySQL errors are occurring, I would expect an error to be logged if the product data couldn't be inserted for a particular reason.

That's certainly how it feels with this problem.

I've just done another couple of rounds of testing. I had error logging enabled for PHP already, but added error logging, general logging, and warnings, for MySQL server.

I als read the entire PHP installation guide at http://us3.php.net/manual/en/install.windows.php including all comments left by others on each page.

This time I enabled PDT again and put another live payment through and once again it recorded everything but no emails and no products.

I've examined the PHP logs and found only an error relating to the DPS/PxPay gateway that runs on one of the live carts. However that gateway works fine anyway - I had to change over to it last year when I found I could not use PayPal.

The MySQL logs aren't showing any problems.

The ZC cache folder only has zcInstallLog files in it.

Is there any way ZC can be made to simply store the order in the orders_products table no matter what PayPal does?

As I mentioned, the DPS payment gateway works perfectly, and so do the other payment systems such as offline credit card and cod, so products are able to be written to that table. Why won't the PayPal gateway do the same?

What is the PayPal gateway even doing? It seems as if it doesn't store the order at all until PayPal has been in touch. Is there a way I can make it store the order no matter what happens at PayPal? That would resolve the problem.

Thanks for any further advice.
Rob

[DrByte]

What is the PayPal gateway even doing? It seems as if it doesn't store the order at all until PayPal has been in touch. Is there a way I can make it store the order no matter what happens at PayPal? That would resolve the problem.

That's an architectural issue, not specific to PayPal, which is addressed in v2.0.0, and too vast to be done in a bugfix release to v1.3.x.

The best workaround to the many limitations of the IPN method is to use Express Checkout which is NOT dependent on waiting for PayPal to be "in touch" as you say.

[robax]

Thanks for the reply.
I'm looking at the PayPal Express payment page here and it looks to me as if it is only meant for payments made from PayPal accounts, not for credit card processing. Is that correct?

I'm only interested in PayPal for credit card processing. Payments through PayPal accounts aren't required. I'm only using PayPal at all because the store operators are set up with that and it's very difficult to move them onto something else. I've managed to get one store onto DPS/PxPay, but the other wants to stay with PayPal.. except it doesn't work.

So if the Express Checkout doesn't do credit card payments then I might just be stuck.
Rob

[DrByte]

Express Checkout can *definitely* be used for making payments via Credit Card. The customer simply fills in their card number on the paypal page when they're redirected there.
Largely the same user experience as with IPN, but without all the technical limitations of relying on postbacks from PayPal.



https://www.zen-cart.com/tutorials/index.php?article=28

[robax]

Thanks for the clarification DrByte

The site is in Australia and I'd need to check that it works like that there, but yes, if it presents the same card/login options as the standard payment page has then that would be just fine. I'll have a closer look into it.

And then I can sleep :)
Thanks heaps
Rob

[DrByte]

Note that v1.3.9b has a few updates to Express Checkout which you will want. I suggest upgrading before testing it out ;)

[robax]

Yes, definitely I'll get that installed as well.
Thanks heaps!
Rob