Performance degrades with many products

[TinaConsultant]

Our site started slowing down awfully as our number of products grew. The product info display page was worst, sometimes taking 8-9 seconds to load. I eventually tracked down the problem and we are back down to 3-4 seconds. Other pages have improved too, simply because the server isn't so busy reading files.

I really wanted to post this as I think zencart is a great product and I would hate to see people abandon it because of this issue.

We have additional images turned on. This module (module/additional_images.php)was reading every single image file in the images/products directory:
while ($file = $dir->read()) {
if (!is_dir($products_image_directory . $file)) {
if (substr($file, strrpos($file, '.')) == $file_extension) {
if(preg_match('/\Q' . $products_image_base . '\E/i', $file) == 1) {
if ($file != $products_image) {
if ($products_image_base . str_replace($products_image_base, '', $file) == $file) {
error_log('I AM A MATCH ' . $file);
$images_array[] = $file;
} else {
error_log('I AM NOT A MATCH ' . $file);
}
}
}
}
}
}

I replaced the above code with:
$searchstr = $products_image_directory . $products_image_base . "*" . $file_extension;
$matches = glob($searchstr);
if(is_array($matches) && !empty($matches)){
foreach($matches as $match){
$images_array[] = basename($match);
}
}

Hope this helps someone else and maybe it can go in the next release.

[lat9]

Gotta love glob! That sounds like a good suggestion.

[mc12345678]

It may also help to use more directories than the simple one directory for all product. Less files in a directory, the less additional reading needs to be done.

There have been some changes to the flow of the file, but those have been made to support image related programs such as image handler and other additional image plugins such as fuelslimbox and colorbox. Changes for such replacement with glob do not appear to have been yet directly suggested, though there have previously been discussions and changes elsewhere to use glob.

Looking over the suggested revision, it does appear that it would still work.

[DrByte]

I replaced the above code with:

        $searchstr = $products_image_directory . $products_image_base . "*" . $file_extension;
        $matches = glob($searchstr);
        if(is_array($matches) && !empty($matches)){
            foreach($matches as $match){
                $images_array[] = basename($match);
            }
        }

Hope this helps someone else

Thanks for posting the suggestion, and for confirming what worked for you and why!
Refactoring to use glob is on the list of things to consider, after resolving some complications that were encountered the first time it was explored.

maybe it can go in the next release.

Hopefully.
And hopefully you'll be upgrading your site soon too ... I note that you're using the 2012 version of Zen Cart, which is kinda dated ...

[TinaConsultant]

Image subdirectories for us would be quite difficult to manage. We have 3000+ products and 7000+ images, with the products being uploaded each night from the main corporate data server.
Subdirectories per product would be hard to keep clean as products are removed. Subdirectories per category might run into problems as we do have a quite a few products that belong to multiple categories.

[mc12345678]

Image subdirectories for us would be quite difficult to manage. We have 3000+ products and 7000+ images, with the products being uploaded each night from the main corporate data server.

So there are processes in place to not only address product "generation" but image upload as well, sounds like a perfect opportunity to do some upfront optimization to support the in operation task(s).

Subdirectories per product would be hard to keep clean as products are removed.

So are you saying that currently you have issues with image removal/recovery? I ask because if this wil be an issue in the future related to product removal, it is now also...

Subdirectories per category might run into problems as we do have a quite a few products that belong to multiple categories.

Every product has one master category. Now if all of your product have the same master category and they are all then placed in some other linked category, well no better off than the current situation of pushing all the images into a single directory (because the same thing would be happening just under a different folder name). If the same product is being pushed to different products_id values then likely causing issues with search engines where you have the same content at multiple destination addresses and likely not canonicallized.

What I'm saying is this in trying to speed up the site: if you have seen an output performance improvement by modifying the code on the back end of retrieval, what amount of improvement do you think you'll have when you improve the level of effort for that process to do anything? How many resources would be spared from that visit/visitor that could in turn be used to help the other guest(s) that are also in need of the same type of operation? Keep seeking code improvements as well, but I can't say that I've heard anything yet that is a business negative to some level of organization up front such that such image location is made faster and I'm not talking about making a directory per product that only has say 6 images perse, but even if the directories were split up to have 10 product in each directory (a little difficult for new product unless the image record were updated after the products_id was known) as identified by say the integer result of products_id/10. Even if the number were say 100, whatever, better than 3000 product with 7000 images in the "folder"...