Internal server error (500), yet pages of store load normally (...)

Hello,

Our Technical Dept. has tried everything, and our last resort is Zen Cart (and the control panel vendor, CPanel) to resolve this.

We have searched all documentation, as well as forums and online resources, and could not trace a problem similar to ours. We have set permissions (755/644/et al) as recommended by Zen Cart documentation for SUPHP installations, as well.

A customer has specified a very weird error on his website, namely he is facing problems with search engine crawlers / bots.

Upon investigation, we found out a very peculiar outcome:

The customer's website ( http://www.futurehouse.gr , based on the last stable version of Zen Cart) loads normally to every human visitor using any browser, such as Firefox, IE, et al.

However, attempting to access this website either as a bot, or even using the 'wget' command, gives out an HTTP 500 Internal Server Error message.

As can be easily understood, crawlers / bots give up after encountering an HTTP 500 error, while for some reason unknown to us, human visitors using conventional browsers can view the website normally, despite the HTTP response, the website loads as normal.

We have found no information at all for this error message in our error logs, but I thought that you might be able to point us to the right direction.

The server runs a relatively hardened PHP installation, under a SUPHP/Suhosin/ et al environment.

The customer, rightly so, complains that his website is totally inaccessible to search engines. Yet it works...with humans, or at least browsers.

Any kind of feedback would be most welcome. We will also contact the control panel vendor about this issue, in case they can point us to a new direction.

Welcome to the Forum. :smile:

It's not uncommon to have 'wget' disabled but it is strange that SE Bots would be denied access. Suggest you check your .htaccess file and chat with your Hoster as they may know more about the Server settings that could cause the Bot problem.

I'm the hoster :P

Customer does not know why the problem appears, and we didn't find any information anywhere which would relate to the problem.

I have never seen something open up normally through browsers and yet deny access through wget or bots/scripts, so if it's not config-related, I don't know what it is. A bug, I'd reckon.

I can't seem to replicate the problem on that site when emulating wget or msnbot, or other common spider engines.

What's the permission level set on /includes/spiders.txt? Is that file damaged in any way?
What Zen Cart version is in use?
What addons are installed? How is this installation different from a brand new clean installation without any addons or customizations using the latest version of Zen Cart?

Quote:

---

The server runs a relatively hardened PHP installation, under a SUPHP/Suhosin/ et al environment.

---

As an IT guy, this is a red flag for me. Without removing the hardening from the equation, we cannot rule out that your server is getting in the way. Would it be possible for you to carbon copy the entire site (files & mysql database) to another location without the PHP hardening? That would definitively pinpoint the issue as residing with the software or the server.

You said you're running "the last stable version of Zen Cart." What version specifically are you running?

Further investigation, which you can do yourself, reveals that your server *is* actually sending a 500 response.

Try this:
- Use Firefox
- Enable the LiveHTTPheaders extension
- Visit the site

You'll see that the first response is indeed a 500 Internal Server error response.

So, wget is legitimately saying what's really going on. Human browser viewing is apparently ignoring that in favor of the rest of the data that follows it.
As the host, you'll need to fix whatever's triggering that from your server.

Code:

---

http://www.futurehouse.gr/



GET / HTTP/1.1

Host: www.futurehouse.gr

User-Agent: Mozilla/5.0 ... Gecko/20100101 Firefox/6.0.1

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip, deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

DNT: 1

Connection: keep-alive

Cache-Control: max-age=0



HTTP/1.1 500 Internal Server Error

Date: Tue, 06 Sep 2011 18:19:26 GMT

Server: Apache

X-Powered-By: PHP/5.2.17

Expires: Thu, 19 Nov 1981 08:52:00 GMT

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Pragma: no-cache

Content-Encoding: gzip

Vary: Accept-Encoding

Connection: close

Transfer-Encoding: chunked

Content-Type: text/html; charset=utf-8

---

More technical detail:
Human-readable web browsers are fairly flexible. Even if they receive an error code, they will still display any page content returned along with the error code response. They only fall back on their default error view if no content is provided.

The wget command and search engine crawlers are far more strict, they will bail out as soon as they see the error code response.

This is definitely a problem with your server configuration.

Let us not confuse 'wget' with 'GET' as they are two different commands.

Although I'm not sure about 'wget', the GET command is working just fine as evidenced by your pages showing. Although there is one 500 error code, it is at the beginning of the page and I suspect is has to do with JS being used. Try removing the "Script Source: CodeLifter.com" code block and see what happens. Otherwise, there could be a clash between the umlauts used within the language and SUPHP which could require a lot more work to correct.

As to the SE Bots being blocked, not sure how you know they are being blocked nor what type error msgs. are being generated. What is different between the main site and the 'test' site; since the 'test' site is being indexed.

Dear all,

Thank you for your feedback so far.

As I said, I do know that the server is sending an HTTP 500 error, and that browsers do ignore it. However, it happens only with ZenCart pages such as this one.

To the best of my knowledge, the customer has installed 1.3.9h, and I have no clue on addons and specific ZenCart configuration used.

I am just baffled by the fact that:

• This particular error does not take place on any other account on the server
• We have found nothing in error logs, either Zencart's or Apache's
• Usually when an HTTP 500 error presents itself, it was displayed in browsers as well, and page loading never happened, unlike what happens here.
• We have transferred the website to a test server using similar (but not 100% identical) configuration options, and it works. However, the test server has no other websites on it, and is generally not a production-used web server.
• Even on the account mentioned (futurehouse.gr), uploading different PHP or HTML files that are not "linked to zencart" do work.
  
  See: http://www.futurehouse.gr/test.php
  
  It appears to me that Zencart, and Zencart alone, has a nudge with something undefined on the server configuration, which causes it to half-work. I can be perfectly sure about this, considering that all other software used (CMS, blogging, e-commerce, CRM, et al) on the machine by other customers, does not have this problem.
  
  It's as Zencart-specific as it gets - And we have found nothing to indicate what could cause it, yet.

The "easy solution" would be to take my customer's websites off this server, and place him in a VPS where we could recompile everything to perfectly match his own configuration (including Zencart). However, this would not indicate to me what could cause this frankly peculiar behavior of having pages load, despite an HTTP 500 error present, with nothing found on error logs.

Finally, I should not that I am not the server admin here - I am the account manager of the customer :)

If you do have some deep technical question, despite my broad understanding, I'd need to relay it to our Tech Dep. But any suggestion would be welcome, really. We have contacted CPanel about this and they indicated that it is an "application or coding issue", thus cannot assist us since it's not a general situation / fault.

Sovjohn,
If the problem is persisting, then, as you say, your research points to the problem existing in your customer's Zen Cart setup.
Now, be careful not to generalize that the problem is actually Zen Cart, but rather it's your customer's *implementation* of Zen Cart

So, your next step should be to install a clean fresh UNCUSTOMIZED copy of Zen Cart onto your customer's account. Test that there. When you see that it's not throwing a 500 error on a clean install, then you'll be able to work with your client to determine which of their myriad customizations has caused them the problem they're experiencing.

As the 500 error is no longer showing within the Headers, looks like disabling "Content-Encoding: gzip" solved that problem.

Sadly, you are correct.

The customer informed me that he isolated the problem in a plugin (probably badly coded one, might I add).

I just tended to believe that this was some deeper issue - which it wasn't.

Feel free to close this thread, since this obviously is not a bug of any sort - And I'd like to thank you all for the discussion, maybe it will prove useful to others experiencing similar plugin related errors! :smartalec:

It can be even more useful if you shared which plug-in was the offender.:yes:

Hello all,
I had the same problem, google just got 2500 error Internal server error 500 and I cant understand why becouse The page display without problem visible.
Well, had made some modifications on tpl_product_info_display.php and these modification crashed something.
I just restore this TPL and problem disapear.