getting a 302 response header for non-existent urls and a redirect to the Home Page?

[piker]

I'm thinking I may have changed a setting somewhere because shouldn't I be getting a page redirect to the Home Page with a 404 Header at the very least? This isn't great for my SEO indexing so I would really appreciate any advice on where to start tracking this down. I'm using Ceon Uri Rewrites, but that seems to be working as it should..
Cheers.

[stevesh]

What happens when you temporarily turn off the URL rewrites ?

[piker]

What happens when you temporarily turn off the URL rewrites ?

I have tried that in the config settings only (no change), but should I also comment out the rewrite rules in the htaccess at the same time?

[DrByte]

Ya, without turning off the plugin and removing the rewrite rules, it's really hard to ascertain whether your problems are a result of the plugin or not.

Plus, a list of problematic URLs would be useful too. Leave off the http:// and any www. when posting them here unless you want them to get reindexed!

[piker]

Ok, when I comment out the rewrites and turn off the rewrite mod I get this:
allnatives.com.au/foo (gets a 404 as it should, instead of a 302)

But strangely I also get a 403 for these (which I don't get when I have the mod working):
allnatives.com.au/images/index.php?main_page=page_not_found
allnatives.com.au/includes/templates/forest_path/css/images/index.php?main_page=page_not_found
*if the mod is on and working I get a 302 for these.

Here is my .htaccess details:

#without these I get a 404 AND a Error Document not found error as well, when the mod is off
ErrorDocument 404 /
ErrorDocument 403 /

RewriteEngine on
# CEON URI REWRITES
# Don't rewrite any URIs ending with a file extension (ending with .[xxxxx])
RewriteCond $1 !\.[a-zA-Z]{2,5}$ [NC]
# Don't rewrite admin directory,editors,cpanel or frontend directories
RewriteCond $1 !^(admin|editors|cpanel|frontend) [NC]
# Handle all other URIs using Zen Cart (index.php)
RewriteRule ^(.*)$ index.php [L]

[DrByte]

Ok, when I comment out the rewrites and turn off the rewrite mod I get this:
allnatives.com.au/foo (gets a 404 as it should, instead of a 302)

Okay. That's to be expected. The mod would do a 302 because of your rewrite rules.
If you want it to give a 404, you'll need to do some work writing more complex rewrite rules.

But strangely I also get a 403 for these (which I don't get when I have the mod working):
allnatives.com.au/images/index.php?main_page=page_not_found
allnatives.com.au/includes/templates/forest_path/css/images/index.php?main_page=page_not_found

That's because of the security protections set in the .htaccess in /images/ and /includes/ to specifically prevent access to .php files, to prevent malicious exploits. So, that's the correct behavior.

*if the mod is on and working I get a 302 for these.

That's fine too, since it still prevents unauthorized access.

#without these I get a 404 AND a Error Document not found error as well

That's to be expected, since a 404 *is* a "Document not found" (aka "Page not found") error.


This all raises a deeper more important question: where are all these invalid hits coming from? If they're from search engine spiders then go clean up the pages where the links were gleaned else they'll just keep coming back no matter what header response is issued.

[piker]

Thanks.
Foo was just an example, there aren't any in particular that i'm trying to fix I just expected to see a 404 and not a 302 for non-existent urls. The Google "problem" (only recently it seems..) is that these 302's back to my Home Page are seen as "soft 404s" and they don't like them much. I don't mind having a crack at a more complicated rewrite rule if anyone has an example?
Cheers.

[lhungil]

I'm thinking I may have changed a setting somewhere because shouldn't I be getting a page redirect to the Home Page with a 404 Header at the very least? ...

There is a Zen Cart setting for how to handle when a page is not found. The options are to redirect to the home page, issue 404 but still show content, or send to a dedicated 404 page. Might want to check this setting.


"Configuration" -> "My Store" -> "Missing Page Check". If this is set to "on" you are redirected to the home page... I typically use the "Page not Found" option and modify the 404 Page built into Zen Cart to fit the store's needs. Using the "Page not Found" option plays well with most of the modules for generating alternative URLs including CEON URI Mapping.

As previously mentioned, if you see 404s in Webmaster tools, probably best long term to fix the links pointing to old / invalid URLs. But hopefully the above info helps as well!

[piker]

"..probably best long term to fix the links pointing to old / invalid URLs.."
The main reason i'm seeing invalid links appearing in GWT all of a sudden (10-15 more a day!) is that a lot of my image files, which I don't mind being indexed, have a name like "foo bar.jpg" (I know I probably should have left that space out!) which are usually encoded for links as "foo%20bar.jpg", but there are some (and i'm suspecting Social Networking sites here) are encoding the link as "foo%2520bar.jpg". People click these and they get a 302 redirect to my Home Page, hence continual "404 soft errors". I think the best option (please correct me if i'm wrong) would be to rename all my images in the style "foo-bar.jpg" and do a 301 redirect to them. Sound good?
Thanks as well for your help, much appreciated.

[lhungil]

sounds easier than dealing with extra rewrite rules to undo the double urlencode you are seeing (another option)... Out of curiousity, is this a specific social media API or network? Just in case anyone else runs into something similiar?