lhungil:
If you cut over the DNS does the lesser VPS still keep up (or does it start to lag)?
Sorry. I don't quite follow. Both sites are using the same DNS. The original site is "www" and the copied site is call "www2"
Not only was this the easiest approach, but I figured this would take DNS out of the equation.
lhungil:
Similiar load on both sites (connections, end users, etc)?
No. The copy site only has me accessing it, so the live site is certainly going to have a heavier load, but certainly not enough to cause a performance issue - And only with some functions??
lhungil:
Same server farm (and network) for both VPS?
No. totally different - intentionally - just in case it is/was caused by some inexplicable networking issue within the server farm.
I'm not sure if I've proven this to be true or not though - It is one of the few things different between the sites.
lhungil:
Same underlying OS / configuration / optimizations? Same PHP <==> HTTP server setup?
As near as possible. All versions are the same on both servers, but I've needed to tweak the mysql settings on the copy server because of the smaller memory. On the live server I have 3Gb 'reserved' just for mySQL, on the copy server it is using the default memory allocations (a lot less).
Hmmm, perhaps I should reset the mysql settings on the live server back to the defaults. I can't imagine this will improve things. Those settings haven't been change since I 1st set this server up about 18 months ago.
[/QUOTE]
Any noticeable pattern looking at page parse / SQL times between the two?
[/QUOTE]
For general page loads, both are on par with each other (the live server being actually a little bit faster - which is pretty much expected considering it has double the grunt).
lhungil:
Is one noticeably slower when writing to the DB via Zen Cart (possibly test using EZ Populate or similiar)?
I don't use EZ Populate or similiar - but again, almost everything about the site(s) are fine - adding/removing from cart, navigating pages, logging on/off, adding/editing products, and so on - The problem only shows with the 'confirm order' (customer purchase) and the 'update status' (admin).
Oh, another finding - I initally said one of the sites had the update problem and the other had the confirm order problem - This turned out to be incorrect - both sites exhibit the same two problems.
One of them (mine) is still using Zen 1.3.9 (I use it mostly for ozpost compatibility testing), the other is using Zen V1.5.4
lhungil:
Any "security" software on either?
Yes, but all have been disabled at various stages of debugging. The problem continued.
lhungil:
Any proxy / cache / CDN setups?
There is obviously caching going on somewhere along the line (local browser, ISP transparent proxy, and obviously? the ImageHandler cache) - but these will be consistant with either site. The servers aren't proxied (that I'm aware of - I'd be piffed if they were). We do use a CDN for both sites and servers, but disabling this didn't change the problem one iota.
lhungil:
Any "limiting" or QOS?
Nothing I've implemented. Who really knows what the hosts have going on behind the VPS.
lhungil:
Any output filters (PHP or HTTPd)?
mod_security (default settings) on both servers - No difference when I've disabled it.
lhungil:
Similiar physical hardware
No way of knowing - they are both VPS's rather than bare metal servers.
lhungil:
for both hosts (CPU, mem, hdd - speed, latency, bandwidth, etc)?
2core vs 4core (4core has the problem)
2Gb RAM vs 5Gb RAM (%Gb has the problem)
Virtualised HDD vs SDD. The HDD is the one with the problems, but there is no disk swapping going on (in fact there is enough memory in reserve on this VPS that I didn't even bother creating a swap file :-)
So, to sum up, yes there are differences between the specs of the live server and the copy/test server - but it is the server with the better specs that is giving the problems, and that wasn't what I was expecting when I copied the site to a less powerful server - I was actually expecting the symptoms/problems to escalate (partly why I chose a less powerful server.
The fact that this less powerful server isn't exhibiting any issues at all has actually thrown a spanner in the works for me - because the intent was after 'proving' the problem still exists on the new server I'd be able to add a few 'traps' into various parts of the code to follow exactly where it is when it hangs. This is a bit pointless now because the new server doesn't have any issues at all.
So, unless I can get a better grasp on what could possible cause this, my only options seem to be some 'heavy duty' debugging on the live site - OR, find another 'permanent' home for the site (I can't/won't leave it on the current test site because it isn't a 'conventional' server/host). It is a 'Droplet' - SAAS (pay for what you use) and being relatively new technology (at least to me) I've no idea what to expect in regards to long term reliability and costs. I must say though, it is an ideal system for testing various servers and configurations.
If you (or any other developer) has never heard of 'Droplets', a little more info can be found here https://www.digitalocean.com/
Thanks for your thoughts. You have prompted me to try one more thing on the live server (set the mySql configs back to the 'small server' default. I really can't see this fixing the problem, but it is one of the few things I've not tried yet.
Cheers
RodG