Is it just me or does ZC have a hard time with special characters? Am I being dense in thinking ZC should allow the user to use special characters and automatically escape them? I'm solely referring to the product names and categories of course. I fully understand and appreciate special character anywhere else should be strictly avoided.

Here's my issue. We've got quite a bit of product(and categories and manufacturers) with commas, apostrophe's ampersand's....etc. We made the mistake of not escaping them in the admin during creation now some (not all), won't allow me to escape them. I'm getting a 400 error when I click preview.

So 1. I obviously need to address that ;)

And 2) is there a preferred method to recover from this?

And finally 3) moving forward, what should and shouldn't I expect from ZC with regards to special characters from the store backend/administering side?

Here's some more info.

Access denied with code 400 (phase 2). Invalid UTF-8 encoding: invalid byte value in character at ARGS:products_description[1]. [offset "8"] [file "/hsphere/local/config/httpd2/modsecurity-core-rules/modsecurity_crs_20_protocol_violations.conf"] [line "67"] [id "950801"] [msg "UTF8 Encoding Abuse Attack Attempt"] [severity "WARNING"] [hostname "familynow.ca"] [uri "/OBSCURED/product.php"] [unique_id "OBSCURED"]

Does this mean I need to tackle this with my host?

Yes, your host would be the smart place to start, since they're the source of those 400-errors.

A true comma or ampersand or apostrophe should not cause any such problems. They're not really anything all that "special".
But, if you're pasting "fancy quotes" such as what MS Word creates, instead of proper normal quotes, those extended characters could cause problems, although I'd expect those problems to be different than what you are reporting.

hm. Well, the only other thing I can think of that could be a factor is that I extended the max # of characters for product and category names. Can't seem to find the post I used as a guide to do that at the moment...Do you think that's relevant here? Most if not all the ones that are giving me grief are long tittles.

I'll chat with my host as soon as they open for the day and see what they say.

Just reporting back....

Here's the relevant portion of a reply I sent to my host in case it's of any use to others who may encounter this.

*Just to add a bit...

This is an issue with products that have a special character in the
title AND the image file name. I've managed to add escape codes to all
products that only had special characters in either the title or image
file name.

When I add the escape code and click save I get a 400 error and the
changes aren't being saved.

This is a plea to relax the mod_security settings so I don't have to
re-create all these products in Zen Cart.

BTW the actual product description section doesn't seem to trigger this.
Only when the product title and image filename contain special
characters does the mod_security setting prevent editing anything about
the product. Also, it's not just the plus symbol. There could be more
but the ones I know for certain are the plus, comma and ampersand.

Thank you in advance for your understanding.

Phil*

So long story short...ZC handles special characters just fine in the above mentioned areas but be very wary of your host's security settings before you opt for fancy editing ;)

...my host is on apache

Second opinion anyone?

Here's the response from my host...

*Hello,

On the server you are on you should be able to disable mod_security in a .htaccess file:

SecFilterEngine Off

You may also want to look at your error_logs. I see a lot of warnings:

[Tue Nov 15 10:22:12 2011] [warn] RewriteCond: NoCase option for non-regex pattern '-f' is not supported and will be ignored.

Thank you...

Closing ticket but it will reopen if you reply with further questions/problems.

Regards,*

The first thing that comes to mind is that this might weaken things. That said, is there something I can do to keep the security tight but allow an exception of some sorts?

Also, while I have your undivided attention :)

What's with this error?

[Tue Nov 15 10:22:12 2011] [warn] RewriteCond: NoCase option for non-regex pattern '-f' is not supported and will be ignored.

The only rewrite rule I know of is the Ultimate SEO mod in the .htaccess -which I regret installing btw. didn't realize it wasn't needed. So I'll gladly remove that if that's the source of that error.

Many thanks!

I've found this[1] but I don't know what POST data is being blocked specifically.

ref.
[1]
some wiki I just found

familynow:

The first thing that comes to mind is that this might weaken things.Well, yes, it will. But if you're only doing it in your admin section (ie: /renamed-admin/.htaccess) then it's less of a risk.> familynow:

That said, is there something I can do to keep the security tight but allow an exception of some sorts?See above.

familynow:

What's with this error?

[Tue Nov 15 10:22:12 2011] [warn] RewriteCond: NoCase option for non-regex pattern '-f' is not supported and will be ignored.That's an invalid rule you've added for URL rewriting.
familynow:

The only rewrite rule I know of is the Ultimate SEO mod in the .htaccess -which I regret installing btw. didn't realize it wasn't needed. So I'll gladly remove that if that's the source of that error.If you're removing the code from the .htaccess, then you need to also remove the addon mod code, lest it create additional problems.

DrByte:

Well, yes, it will. But if you're only doing it in your admin section (ie: /renamed-admin/.htaccess) then it's less of a risk.
Hm. odd, I just realized my htaccess file under the admin disappeared...That can't be good. I just copied it from backup now. As soon as I turn off mod_security the page goes blank.

DrByte:

That's an invalid rule you've added for URL rewriting.
I double checked and the htaccess content in my root is stock from the USU mod. See below.

All other pages

Don't rewrite real files or directories

RewriteCond %{REQUEST_FILENAME} !-f [NC]

DrByte:

If you're removing the code from the .htaccess, then you need to also remove the addon mod code, lest it create additional problems.
This is a different topic altogether but I've been meaning to ask...Could you be so kind as to elaborate a bit on whether or not an SEO add-on is actually necessary? I'm getting mixed signals.

hehe. Isn't it always the way...I've been beating my head against a brick wall; my host no longer allows turning off sec_mod in htaccess files :) I'm working with them to have an entry in my apache config for my virtual host.

I miss VPS....

Alrighty...as it stands, this[1] allows me to add/edit products once again. However, this raises security concerns since it turns sec_mod off site-wide. I'm still working with my host to merely allow an exception for domain.com/admin/product.php

ref.
[1]
<IfModule mod_security2.c>
SecRuleEngine Off
</IfModule>

Work with them to wrap a filesmatch directive around it, and/or stick it only in the /admin/.htaccess so that it'll only affect the /admin/ folder, and not the entire site.

familynow:

This is a different topic altogether but I've been meaning to ask...Could you be so kind as to elaborate a bit on whether or not an SEO add-on is actually necessary? I'm getting mixed signals.
Correct, it's entirely off-topic.

And, strictly-speaking, no, you don't need a so-called "SEO" add-on if you write smart content.

Well. There you have it...

There is all kinds of information out there showing different ways to accomplish this and a lot of it really depends on how the web server is set up but basically adding the following, or including it using an Include statement in a vhost configuration file should work.

<LocationMatch '/admin/'>
SecRuleRemoveById 950801
</LocationMatch>

The ID was obtained from the mod_security error which was being logged. At least in your case the example above was included only for the admin directory of your web site.

DrByte:

Yes, your host would be the smart place to start, since they're the source of those 400-errors.

A true comma or ampersand or apostrophe should not cause any such problems. They're not really anything all that "special".
But, if you're pasting "fancy quotes" such as what MS Word creates, instead of proper normal quotes, those extended characters could cause problems, although I'd expect those problems to be different than what you are reporting.

This thread is old, and a lot of changes have been made to browsers and zencart, but I am replying to this thread to avoid creating a new one (too may already, and sometimes it's hard to get answers due to too may results to read through)...

I have a few thousand products (no linked, no options, no attributes) and I use EP4. I would like to add 2 new product lines. 1 has a decimal in all the mfg numbers (v_products_model and v_products_name), and the other has ¼, ½, ¾ in the mfg numbers (v_products_model and v_products_name).

I tried to import them, but the fractions did not work. I then escaped them , and everything seems to work fine.

My question is, though, should I expect to see other problems? For example, I did not test actual purchasing a product yet, with inventory subtraction, etc.

I would really like to keep the mfg numbers as they actually are, BUT I do not want unforeseen problems neither.

Help please! Thanks!

stiggybaby:

This thread is old, and a lot of changes have been made to browsers and zencart, but I am replying to this thread to avoid creating a new one (too may already, and sometimes it's hard to get answers due to too may results to read through)...

I have a few thousand products (no linked, no options, no attributes) and I use EP4. I would like to add 2 new product lines. 1 has a decimal in all the mfg numbers (v_products_model and v_products_name), and the other has ¼, ½, ¾ in the mfg numbers (v_products_model and v_products_name).

I tried to import them, but the fractions did not work. I then escaped them , and everything seems to work fine.

My question is, though, should I expect to see other problems? For example, I did not test actual purchasing a product yet, with inventory subtraction, etc.

I would really like to keep the mfg numbers as they actually are, BUT I do not want unforeseen problems neither.

Help please! Thanks!

If you're asking whether changes to products_model and products_name will affect the math done on the products_quantity fields, I'm not seeing the connection. The math isn't done to the products_model or products_name field.
I suppose if you're concerned that any searches you do based on name or model might be affected, well, yes they will ... but that all depends exactly how you are using those, and whether your customers will ever do things based on those. In the strictest sense it's no different than spelling a word differently: if changing the spelling of something will make perfect sense to you and your customers, then fine. If it won't, then, well, maybe that's not a good idea.

If you're asking about whether you need to do something differently with EP4, best to ask in the EP4 support thread.

DrByte:

If you're asking whether changes to products_model and products_name will affect the math done on the products_quantity fields, I'm not seeing the connection. The math isn't done to the products_model or products_name field.
I suppose if you're concerned that any searches you do based on name or model might be affected, well, yes they will ... but that all depends exactly how you are using those, and whether your customers will ever do things based on those. In the strictest sense it's no different than spelling a word differently: if changing the spelling of something will make perfect sense to you and your customers, then fine. If it won't, then, well, maybe that's not a good idea.

So basically you are saying using quarter, half, and three quarter fractions in some products products_model and that products products_name fields is perfectly okay, everything will work a-ok, problem free providing the fraction special character is escaped, and using a decimal in some products products_model and that products products_name fields is also perfectly okay, everything will work a-ok, problem free and it does not even need to be escaped (and by "a-ok, problem free" we're referring to the internal code functionality of zencart, not desired search results due to spelling or a customers ability or lack of being able to "type" in a fraction character)?

Which basically, in other words, you are saying zencart will have to problems with that, so go ahead and use decimals in those fields if you want, and go ahead and use fraction special characters in those fields too, just escape the fractions and keep in mind potential customers don't have fraction keys on their keyboards?

Thanks again