Default 'extra info' email content triggering gmail spam detection

[neekfenwick]

Goodbye zen cart forum.

[dbltoe]

Must've been what Scott said, right guys?

[swguy]

I have that effect on people.

[davewest]

Did I over do it with the pinky and the brain comment about Gooo...

[swguy]

It should be a point of pride for all contributors that the Zen Cart forum is, for the most part, a friendly and welcoming place.
(Especially compared to other forums, which will not be named, although they rhyme with Shmesta-Shop.)

[mc12345678]

Whatever the case for departure, I would say that if the ip-address and/or hostname were an issue, that they could still be included in the email perhaps with some massaging.

For example instead of the ip address being four groups of information separated by three periods that the period have a space before and aftewr it and any link coded into the numbers just be removed or similar. Or perhaps (haven't looked at the detail in a while) some additional htmlencoding be applied...

Unfortunately there could still be some other reason that Gmail (even when using the g-suite portion) is flagging the email. We haven't even really talked about the settings that have been applied... smtp? Smtpauth? Whatever the new Gmail server option is? All/each of these provides different information to the servers along the way. Last time I looked at using the g suite for a domain, there seemed like a bunch of things needed to be done to have the email come/go through their servers related to dns records. Sounds like those may have all be addressed, but never hurts to ask or look again from the perspective of trust by the server(s).

Have you tried setting up an email address that would be used for sending such messages but perhaps not used as the recipient as well? Wondering if part of the issue is the receiver is the sender type thing...

Overall, doesn't surprise me that messages with more detail that a basic shopper gets when making a purchase get "treated" differently.

[DrByte]

One common tactic used by spammers (er, those who have hijacked a legit mail server) is to send emails "from oneself, to oneself". I can see where spam-trappers could flag that up if other heuristics were a match. Maybe in this case the "other" patterns are the presence of these ip/host details.

Thus, another approach you could try is (as hinted at in one of the comments above) to set up a separate mailbox for contact-us messages, so the send-from and send-to aren't the same anymore, in the case of contact-us anyway. Helpful to try as a test to rule out whether that's a notable contributor to the issue you're having. You might then also send all your admin-copy-of-order-emails to something different than the send-from as well.

As to ripping out the ip/host details from the extra_info section, sure you can do that. As discussed maybe removing the whole section is overkill.

[neekfenwick]

Recently had issues with deliverability to gmail addresses and getting DMARC setup correctly was thing that fixed it.

Thanks for the tip. You may have a point as we don't have DMARC set up. I'd hoped that DKIM and SPF would do it, and to be honest I found the DKIM documentation pretty hard to follow. I'll have another bash at it, and try re-introducing the Extra Info content to see if the detection is still triggered.

[neekfenwick]

Sorry, your reply rubbed me up the wrong way the other day and I responded over the top, apologies for that.

listen, if you got your email working, great. but i submit to you and others here that ip address and host address information is valid information in an email, and one can send ham emails with that information. i do it all the time.

Yes, I do think ip/host is valid content, and we've used the default ZC behaviour for 10 years, moving to gmail about a year or more ago, with no problem. Suddenly in the past couple of weeks we're getting that same kind of content flagged as spam. So either Gmail have changed their spam detection algorithm, or the hostnames being included have changed to include content that trigger the old detection algorithm, or something else is going on. Google aren't willing to disclose any info about their algorithm, at least to me

how would you know? did you contact everyone of them?

I said the emails are not 'classified as spam' because, for the ones that we believe are (the Contact Us ones), we immediately get an email from Google with the subject 'Delivery Status Notification' saying it was blocked, and we don't see that behavious with any of the (far more voluminous) normal order emails. Certainly, some emails to customers may be being marked as spam, but the real indicator for me was that after a series of the reports that we do get, our gmail account is blocked (and we have to go and hit the 'Restore gmail' button), then, after making the change to the Contact Us email content, we haven't had to do that once, despite still sending hundreds of normal order update (and all the other types) of email from ZC each day.

the fact that you could not get exim configured properly and ended up on RBL lists is on you and your host. and if you are using a shared host, well it could be even harder based on who your host is. running exim4 on a debian host and getting that configured correctly, is to me, far preferable to using gmail for sending your email from your website.

All fair points, and I didn't want to get into a discussion about it (it's a whole other subject) I just wanted to be clear that we have tried that approach, i.e. I wasn't just ignoring the suggestion to try doing that.

in your previous setup, you got put on some RBLs, because you and your host could not get exim setup properly. now gmail is telling you, you can't use our SMTP server to send spammy email. looks like the same problem; you have just moved it to a different point in the email chain.

exim config certainly is complicated and we had a series of issues that were years ago now and I can't remember precise details. As I recall we got put on RBLs because our host was compromised and used as a relay to send actual spam email by spammers. Once our domain name was blacklisted it was quite painful to work out of that situation. It wasn't the same situation as this one of ZC content being considered spammy. I could rant on about that and past mistakes but it's not relevant here. We moved to gmail partly because we thought it would remove all possibility of being considered spammy, so it's a kind of dark humour that we've run into this position

email is hard. and i'm not disputing that what you are doing is working for you. but i think the setup is less than ideal, as again gmail is telling you your email to yourself is spammy. and if you want IP information in that extra info email, who the heck is gmail to tell you you can't have it? especially after you are paying them?

Yes, if I could dictate to gmail what is spam and what isn't, life would be great, but even if I pay for a business account we have no actual control over their spam filters. One suggestion to fix the situation is certainly to change email provider (or self-host as you suggest).

If your overall suggestion is to 'not use gmail' then that's fine, I was posting to try to help those who _are_ using gmail and have suddenly found their accounts being blocked in this way. The more I read here, it sounds like the issue on gmail is not considered something that would warrant a core change to ZC (i.e. a Bug), though I haven't yet tried the final step of setting up DMARC in our DNS records, I'll have a bash at that and try to report back.

[neekfenwick]

One common tactic used by spammers (er, those who have hijacked a legit mail server) is to send emails "from oneself, to oneself". I can see where spam-trappers could flag that up if other heuristics were a match. Maybe in this case the "other" patterns are the presence of these ip/host details.

Yes I agree, one of my first thoughts was the pattern that the Contact Us email actually sends an email 'to' us, 'from' us but with a 'Reply-To' header of the customer's email address (for easy Reply action), which I thought could be considered a kind of hi-jack attempt. But this idea was fairly quickly quashed by the Extra Info thing.

As to ripping out the ip/host details from the extra_info section, sure you can do that. As discussed maybe removing the whole section is overkill.

Fair point, I've now re-instated almost everything except the ip/host details, and emails are still coming through fine. (I also took out the disclaimer and footer content as that didn't seem useful for our staff, though now I think about it perhaps it should stay in because when staff Reply to the email, that content is no longer included in the content they receive...)

Of course we're all busy, this is all around a very busy normal day job, I do appreciate the replies.