Site running slowly.. but parse times are good

I'm pretty sure it's because I'm on a shared server with my webhost (Dreamhost), but I want to go to them with as much info as I can so I'm checking here first.

My site (http://www.griffindyeworks.com/store) has been running intermittently slowly for months. Usually just in the evenings PST, but lately all the time.

I followed the directions in the FAQ/Tutorial for optimization. My images are NOT optimized (working on it), but since it's slow even in the admin I don't think that's the cause.

Parse Times:
Main page: Parse Time: 0.475 - Number of Queries: 156 - Query Time: 0.077134163742065
SubCat page: Parse Time: 0.528 - Number of Queries: 109 - Query Time: 0.13509468765259

I just tried to get Server Info from the admin and got a 504 Gateway Time-out. Meanwhile, all my other DH hosted sites on the same SQL server are running fine, including Wordpress, Drupal, and Gallery3... AND going back into the admin and trying to access other functions works, albeit slowly.

Any ideas on how to fix or further debug this?

theresab:

I just tried to get Server Info from the admin and got a 504 Gateway Time-out.
<snip>
Any ideas on how to fix or further debug this?

Fixing 504 errors - general
This problem is entirely due to slow IP communication between back-end computers, possibly including the Web server. Only the people who set up the network at the site which hosts the Web server can fix this problem.

Reference:
http://www.checkupdown.com/status/E504.html

Cheers
Rod.

ps. A Gateway error is atypical for a 504 error. It suggests a proxy server is being utilised between the two endpoints.