[Not a bug] HTTP 404 for spiders with Prevent Spider Sessions enabled

[TheBigAristotle]

Hi all,

I'm using the latest version of Zen Cart (1.3.9h) and just found some issues with spiders. When I review the apache log on my server I see all GET's by spiders (Google, Bing, Yahoo) returned with a 404. When I try the "Fetch as a Google Bot" in webmaster tools it also receives a 404. This behavior stops when I disabling the setting "Prevent Spider Sessions".

The problem with disabling this setting is that every URL is appended with the zenid=<sessid> tag which is not how you want search engines to index your site. As a temporary workaround I disabled the setting "Prevent Spider Sessions" and modified the function zen_href_link in so it no longer adds the zenid parameter when the user agent is a spider.
This is not really a good solution as in the Who's Online page the search engines are no longer highlighted and this is solving the symptoms, not the issue. So that's why I'm opening this topic right now.

I tried to narrow down the source of the 404 to the code related to the prevent spider session in init_sessions.php by commenting out the following chunk:

 if (isset($_GET['zenid']) && $_GET['zenid'] != '') {
      $tmp = (isset($_GET['main_page']) && $_GET['main_page'] != '') ? $_GET['main_page'] : FILENAME_DEFAULT;
      @header("HTTP/1.1 301 Moved Permanently");
      @zen_redirect(@zen_href_link($tmp, @zen_get_all_get_params(array('zenid')), $request_type, FALSE));
      unset($tmp);
      die();
    } 

This has no influence in the behavior.

When I copy the URL in the apache log to my browser I get the page I want so it has to be some error in the script caused by the Prevent Spider Session setting.

Any ideas where this behavior comes from?

[DrByte]

I can't replicate the problem you're alleging.

What's the exact URL you're complaining about?

[TheBigAristotle]

Hi,
Thanks for your reply. This issue exists for all the pages on my zen cart store, not just a few off them.

This is what I get when I use the fetch as a google bot tool on the google webmasters website:

HTTP/1.1 404 Not Found
Date: Wed, 24 Nov 2010 20:28:13 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: zenid=sessid; path=/shop; domain=.www.domain.tld; HttpOnly
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 20
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

[TheBigAristotle]

Hi guys,

I started to debug this further and apparently this was caused by a modification I made. I improved some checks regarding sessions and now this is no longer a problem.

Thanks anyway for your help!

Best regards