Random Malformed order confirmation emails

Hey everyone

We're a charity choir in the UK using ZenCart version 1.3.8 - http://tickets.elmbridgechoir.co.uk/zencart/index.php

Shop works great, no add-ons or customisation other than template and a few other small wording thingies.

We're using TEXT based order email confirmations.

One intermittent problem we seem to have is that the odd confirmation email is malformed in that it randomly ends the email after the "Products" section as per this example:

" -----Original Message-----
From: Elmbridge Choir Ticket Shop [mailto:[email protected]]
Sent: 05 July 2009 07:59
To: [email protected]
Subject: [NEW ORDER] Order Confirmation No: 111

AuthCode: TransID: 71P1129940060464T

Order Confirmation from Elmbridge Choir Ticket Shop

CUSTOMER NAME

Thanks for shopping with us today!
The following are the details of your order.
------------------------------------------------------
Order Number: 111
Date Ordered: Sunday 05 July, 2009
Detailed Invoice:
http://tickets.elmbridgechoir.co.uk/zencart/index.php?main_page=account_history_info&order_id=111

Products
------------------------------------------------------
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £16.00
Ticket Type ADULT
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £8.00
Ticket Type CHILD"

I've checked the email archive and there seems to be no pattern. The only possible connections are:

• Always PayPal payment
• Two line items (different attributes) for the same product as per above example

Checking in the database email archive, our admin copy of the email looks as per the example above, the email that went to customer, looks like this:

"Order Confirmation from Elmbridge Choir Ticket Shop

CUSTOMER NAME

Thanks for shopping with us today!
The following are the details of your order.

Order Number: 111
Date Ordered: Sunday 05 July, 2009
Detailed Invoice: http://tickets.elmbridgechoir.co.uk/zencart/index.php?main_page=account_history_info&order_id=111

Products

2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £16.00
Ticket Type ADULT
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £8.00
Ticket Type CHILD

This email address was given to us by you or by one of our customers. If you feel that you have received this email in error, please send an email to [email protected]
This email is sent in accordance with the Privacy and Electronic Communications (EC Directive) Regulations 2003. Removal requests can be sent to this address and will be honored and respected."

Our normal admin confirmation email looks like this:

"-----Original Message-----
From: Elmbridge Choir Ticket Shop [mailto:[email protected]]
Sent: 29 June 2009 13:51
To: [email protected]
Subject: [NEW ORDER] Order Confirmation No: 103

AuthCode: TransID: 9NS076642W350091R

Order Confirmation from Elmbridge Choir Ticket Shop

CUSTOMER NAME

Thanks for shopping with us today!
The following are the details of your order.
------------------------------------------------------
Order Number: 103
Date Ordered: Monday 29 June, 2009
Detailed Invoice:
http://tickets.elmbridgechoir.co.uk/zencart/index.php?main_page=account_history_info&order_id=103

Products
------------------------------------------------------
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £16.00
Ticket Type ADULT
------------------------------------------------------
Sub-Total: £16.00
Collection in Person (Collect at choir rehearsal): £0.00
Total: £16.00

Delivery Address
------------------------------------------------------
Address removed
Address removed
Address removed
Bookham
Surrey
ZipCode re=moved

United Kingdom

Billing Address
------------------------------------------------------
Address removed
Address removed
Address removed
Bookham
Surrey

ZipCode re=moved

United Kingdom
Payment Method
------------------------------------------------------
PayPal

-----
This email address was given to us by you or by one of our customers. If you
feel that you have received this email in error, please send an email to
[email protected]

-----
Copyright (c) 2009 Elmbridge Choir Ticket Shop. Powered by Zen Cart

Office Use Only:
From:
Email:
Login Name: Customer name
Login Email: [email protected]
Telephone: 01372 450551
IP Address: 91.85.111.111 - 91.85.111.111
Host Address: 91.85.1111.111
Date and Time: Mon Jun 29 2009 13:51:08 BST"

This is totally random and the section that i9s always missing when it does happen is from the "sub-total" section down to the "This email was given to us..." section.

Does anyone have any ideas why this could be happening, please?

Thank you so much

Cliff

UPDATE: The only other common denominator between the malformed order emails are that they are all "PayPal Express Checkout (instant)" payment methods - not sure if this means anything, but nevertheless.

Cliff

Hello!

Is there anyone that can help with this? I believe that this may be a hidden bug in Zencart - I have now figured out that it happens for EVERY paypalwpp instant payment where the following condition is present:

• there are TWO line items of the SAME items on the order but with different attributes
  example: 2 x ITEM 1 Attribute "a" plus
  2 x ITEM 1 Attribute "b"

This is really serious for our live shop as the email is malformed and not all information is provided to the customer as confirmation of their purchase.

For an order with the above conditions, the email terminates prematurely as follows:
*""Order Confirmation from Elmbridge Choir Ticket Shop

CUSTOMER NAME

Thanks for shopping with us today!
The following are the details of your order.

Order Number: 111
Date Ordered: Sunday 05 July, 2009
Detailed Invoice: http://tickets.elmbridgechoir.co.uk/...o&order_id=999

Products

2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £16.00
Ticket Type ADULT
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £8.00
Ticket Type CHILD
************* TERMINATES HERE, BUT ADDS THE FOOTER BELOW **********
This email address was given to us by you or by one of our customers. If you feel that you have received this email in error, please send an email to [email protected]
This email is sent in accordance with the Privacy and Electronic Communications (EC Directive) Regulations 2003. Removal requests can be sent to this address and will be honored and respected."

Anyone, please?

Cheers

Cliff

I'm having difficulty replicating the situation on a clean 1.3.8 install ...

What other payment modules do you have enabled?
Is it in any way isolated to certain products? or just the combination of the same product listed twice with different attributes?

Hey there, thanks for taking the time to get back to me. I have the following payment modules installed:

• free charger
• money order/check
• PayPal IPN WPS
• PayPal Express Checkout

All the others work well, as does Paypal for one or multiples of different products.

The issue only occurs for module paypalwpp (PayPal Express Checkout (instant) ) and ONLY if customer chooses 2 line items with different attributes (in our case example 2 x tickets for a concert for attribute ADULT and then 2 x tickets for the same concert for attribute CHILD.

The above condition happens for ANY product, not just a specific one, which leads me to believe it's a consistent bug.

See my shop at tickets.elmbridgechoir.co.uk

Perplexing indeed!

Cheers

Cliff

Definitely perplexing. I can't make that happen no matter how hard I try.

This is where I usually start when encountering such perplexing things ... http://www.zen-cart.com/wiki/index.php/Troubleshoot_-_Diagnosing_Obscure_Issues

Dr. Byte, thanks for this very useful suggestion!!

The only meaningful differences that I can find are:

A. Interestingly enough, in the paypalwpp.php file. I seem to have the "11302007" full release (3.8) installed but I have compared my install to the "12112007" version (3.8a)also available.

Here are the differences, seems to be mostly to do with debug mode though... do you think this could cause the issue and if so, can I safely replace my version with the 12112007 version of the file?

*<snipped by moderator>
*

Look forward to hearing from you whether you think any of these could be causing this?

Cheers

Cliff

I would use the more recent of the two versions.
Note that v1.3.8a is newer than v1.3.8, and has a couple tiny but important bugfixes in it.

None of the things you mention would have any bearing on order confirmation emails.

Hallelujah!! After weeks of searching and struggling, I FOUND THE ERROR!!

Turns out that it had nothing to do with PayPal, it was to do with an OPTION VALUE (DESCRIPTION)!

One of my option descriptions were "CHILD <12"

When Zencart produces the TEXT email, it terminated the email when it encountered the "<" in the option description, but works fine in the HTML version. The copy of the email going to store owner always terminated on that option due to that email always being text.

This must be A BUG, THE "<" is contained in the VALUE FIELD of the Option and should be somehow be converted to text when creating a text version of the email, perhaps by using "<"?

Hope this helps anyone else that encounters the same problem. Once I changed the Option value to "CHILD under 12" it works fine!!

Cheers

Cliff

The < character is probably being treated as the beginning of an HTML tag, and since HTML tags are stripped out of text-only emails (for safety and appearance reasons), that's likely why the problem is occurring for you.

Glad you have a workaround.

Yes thanks! I guess this should be reported as a bug since, if the character is encountered as part of a static description field, it's obviously not an HTML tag and should be preserved for the text version.

Cheers

Cliff

We'll look into it.

However, HTML tags do (unfortunately) get used in various description fields, making it a little tough to apply simple generic rules to data parsing. Especially when the 'value' could be a text field that was entered by the customer ... which then brings security risks if not properly sanitized.

Puzzling. I've tested the use of "Child < 12" as the option value, and can't reproduce any problems with emails getting truncated.

Dear Dr Byte

I can now confirm that it ONLY happens if this exact condition is met : "Child <12" i.e. if there is NO SPACE between the "<" and the next bit of text! How funny!!

I made a brand new test install and tested "Child < 12" which worked fine, and then "Child <12" which malformed the email.

Can you work that one out? :)

Kind regards

Cliff

Hah! I also notice that if you should enter "Child <Twelve" for example, then it totally ignores the "<Twelve" bit in the description in the Admin panel, and in the shop it shows "[Child <Twelve"]" (i.e. followed by a quote mark)
See image below:

Interesting!

What happens if you replace the "<" with "<" instead?

Dr Byte

Just tested that, and whilst it looks correct on the option value (shows Child <12", when it gets to the checkout cart, it shows as "Child <12" and the email is once again malformed.

Cheers

Cliff

What exactly do you mean by "the email is malformed"? You mean it skips a section? Or do you mean the < shows up in the email and makes it look funny?

I showed in my first post exactly what happens. The email terminates after the "Child <12" product line is encountered and it skips the rest of the email - leaving out the order totals, payment method, shipping details etc.. here is the example from the first post again:
"-----Original Message-----
From: Elmbridge Choir Ticket Shop [mailto:tickets@*****.co.uk]
Sent: 05 July 2009 07:59
To: [email protected]
Subject: [NEW ORDER] Order Confirmation No: 111

AuthCode: TransID: 71P1129940060464T

Order Confirmation from Elmbridge Choir Ticket Shop

CUSTOMER NAME

Thanks for shopping with us today!
The following are the details of your order.
------------------------------------------------------
Order Number: 111
Date Ordered: Sunday 05 July, 2009
Detailed Invoice:
http://[removed]

Products
------------------------------------------------------
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £16.00
Ticket Type ADULT
2 x Summer Picnic Concert - Molesey Regatta 19 July 2009 = £8.00
Ticket Type CHILD"

It terminates here and only adds the " This email address was given to us by you or by one of our customers..." bit.

It also does the same with the email that goes to the customer.
Cheers

Cliff