Corrupted product database entry leads to blank product listings

Perhaps delia can share a link to a (zip of a) "too large image file" that can be used for testing on other servers?

That's doable. Here's one that was actually in use on this site that I was posting about. http://zencartdelia.com/large_image.zip

Following up to see if the changes posted to the Github repo resolved the reported issue.. I'd like to submit the update IH4 to the Zen Cart free downloads..

I haven't even had time to check, sorry! Will do asap.

delia:

I haven't even had time to check, sorry! Will do asap.

Sure.. I made the same updates to my client's site, and so far no issues.. but I need confirmation from an independent source that lhungil's changes did the trick..

Hate to be a pest about this, but I would like to submit the IH4 updates from my repo to the Zen Cart downloads repo, and I need someone other than just me to give it a thumbs up..

Delia.. have you had a chance to download and install IH4 from my Github repo, and try it on your client's site?? I need to confirm that lhungil's suggested modifications solve the issue you had..

The client's website issues were solved by reducing the size of the images so I don't have an easy way to test. Otherwise I would have been digging in on this long ago. I'm going to have to set up a new test site with one of the images that I think will trigger the issue. "think" because I have to do this on my own server and I don't know if this is a problem on it or not as I haven't run into it in a while and my server has changed somewhat. You just plain weren't able to replicate the problem?

delia:

The client's website issues were solved by reducing the size of the images so I don't have an easy way to test. Otherwise I would have been digging in on this long ago. I'm going to have to set up a new test site with one of the images that I think will trigger the issue. "think" because I have to do this on my own server and I don't know if this is a problem on it or not as I haven't run into it in a while and my server has changed somewhat. You just plain weren't able to replicate the problem?
Oh yes, I think I acknowledged a post or two back that VERY large image files will present an issue, and may lead to the result you reported.. but this is likely a server out of memory situation.. The "fix" I posted was based on lhungil's suggestion to make some changes so that some of the error messages that are NOT displaying ("out of memory") will display so we can confirm that this IS what is happening..

Still doesn't solve the issue on FIXING this, but at least we can confirm/see what error logs are created from large images failing to process are due. This may lead to a fix.. (which is still likely over my head to address..:laugh:)

Oh, I see. Well, I've gotten backed up with work and have to tend to that first. I'll let you know.

delia:

Oh, I see. Well, I've gotten backed up with work and have to tend to that first. I'll let you know.

No worries.. just wanted to get some feedback from another source.. I'll probably submit the new version anyway.. I think it will be fine..

DivaVocals:

... The "fix" I posted was based on lhungil's suggestion to make some changes so that some of the error messages that are NOT displaying ("out of memory") will display so we can confirm that this IS what is happening ...
:D Was gonna ask "What fix?", but now I know what you were talking about

Once verified the cause is an "Out of Memory" issue...

It appears (from some quick online searches), imagecreatetruecolor() uses approximately 5bytes per pixel and imagepng() uses approximately 4bytes per pixel. So one possibility would be to check the dimensions of the uploaded images and determine how much memory is needed... Then compare against something like "convertToBytes(ini_get('memory_limit')) - memory_get_usage()" to see if the image will be too high of a resolution to be handled by gd2 in the current environment.

One downside, is there is no simple way to determine from the Zen Cart admin the maximum amount of memory used by a page on the catalog where images will be displayed. This leads to the following two thoughts...

When the maximum memory would be exceeded on the Zen Cart catalog side: display the original image to the end users (or a dummy image). And of course, send an email to the Zen Cart administrative account to report to the admin user(s) the image needs to be replaced with a lower resolution copy (possibly with a calculated max resolution) or contact their hosting provider to increase the amount of memory available to a PHP script.

When the maximum memory would be exceeded on the Zen Cart admin side (or a configured amount of memory would be exceeded): report back to user they need to make the image resolution smaller (possibly with a calculated max resolution) or contact their hosting provider to increase the amount of memory available to a PHP script.

NOTE: Another option for those who may run into this frequently (due to either very large images or lack of memory on shared hosting) would be to use commandline tools such as ImageMagik (if available on the host). Not my favorite method of handling things, but may be a workaround.

lhungil:

When the maximum memory would be exceeded on the Zen Cart catalog side: display the original image to the end users (or a dummy image). And of course, send an email to the Zen Cart administrative account report back to user they need to make the image resolution smaller (possibly with a calculated max resolution) or contact their hosting provider to increase the amount of memory available to a PHP script.
IMHO, this should be nipped in the bud at the initial image upload. I see a few issues with this suggested course of action.. Displaying the ORIGINAL oversized image is NOT a good idea IMHO.. Depending on the image size, it could grind the site load to a halt.. Also I can see all of these options becocming a NIGHTMARE from the standpoint of supporting this module since there WILL be shop owners who will expect that some level of support in working with their hosting providers.. (not something that anyone supporting IH4 should be getting into in any depth IMHO)

lhungil:

When the maximum memory would be exceeded on the Zen Cart admin side (or a configured amount of memory would be exceeded): report back to user they need to make the image resolution smaller (possibly with a calculated max resolution) or contact their hosting provider to increase the amount of memory available to a PHP script.This to me is the best option.. stops the issue at the onset..

That said.. the changes needed for either the catalog or admin side solution to large image uploads is over my head to execute.. :laugh:

lhungil:

NOTE: Another option for those who may run into this frequently (due to either very large images or lack of memory on shared hosting) would be to use commandline tools such as ImageMagik (if available on the host). Not my favorite method of handling things, but may be a workaround.IH4 has supported ImageMagik (if available on the host) since it's inception.. I'll admit that I've not tested this in quite some time, but to my knowledge IH4 can still process images using ImageMagik.

... IH4 has supported ImageMagik (if available on the host) since it's inception.. I'll admit that I've not tested this in quite some time, but to my knowledge IH4 can still process images using ImageMagik.
Yup, and using ImageMagik (or other commandline tools) may be the simplest solution for stores who experience the "Out of Memory" issues often. Should work, as long as their hosting provider has ImageMagik installed and PHP is allowed to execute the ImageMagic binaries.

I would prefer sticking with gd2 (present in most modern PHP hosting environments). I agree checking the resolution when uploaded via the Zen Cart administrative interface would be the most ADMIN USER friendly method.

It has been awhile since I have looked at IH4, but I remember the ADMIN USER being given the option to generate optimized images immediately after upload... Electing to not generate the optimized images upon upload would force IH4 back to generating those images "on the fly" when the corresponding customer facing page is viewed. The "on the fly" method is also used when the cache is cleared or an image has been manually uploaded...

The amount of available memory on catalog pages will be different from the amount of available memory available on a specific admin page... So how should we handle the "on the fly" image generation IH4 does (when the optimized image has not already been generated)?

Do we just disable "on the fly" (not the most friendly solution)?

If largest is not possible, use / generate medium? If medium is not possible, use / generate small? If small is not possible, use original?

Some other solutions / thoughts?

lhungil:

Yup, and using ImageMagik (or other commandline tools) may be the simplest solution for stores who experience the "Out of Memory" issues often. Should work, as long as their hosting provider has ImageMagik installed and PHP is allowed to execute the ImageMagic binaries.nods in agreement:yes: When I have a minute, I have been giving some thought to adding an admin option to activate using the ImageMagik binaries. Right now it requires users to edit an IH4 file, and the "instructions" on how to do so is a little obscure and hidden.. I think that Tim may have left it this way so that folks didn't cause issues all willy nilly by turning on ImageMagik and the proper binaries were not installed..

lhungil:

I would prefer sticking with gd2 (present in most modern PHP hosting environments). I agree checking the resolution when uploaded via the Zen Cart administrative interface would be the most ADMIN USER friendly method.nods in agreement:yes:

lhungil:

It has been awhile since I have looked at IH4, but I remember the ADMIN USER being given the option to generate optimized images immediately after upload... Electing to not generate the optimized images upon upload would force IH4 back to generating those images "on the fly" when the corresponding customer facing page is viewed. The "on the fly" method is also used when the cache is cleared or an image has been manually uploaded...Optimized images are always generated on the fly.. The option I think you're referring to is the ability to turn off IH4 and all it's features all together..

lhungil:

The amount of available memory on catalog pages will be different from the amount of available memory available on a specific admin page... So how should we handle the "on the fly" image generation IH4 does (when the optimized image has not already been generated)?

Do we just disable "on the fly" (not the most friendly solution)?

If largest is not possible, use / generate medium? If medium is not possible, use / generate small? If small is not possible, use original?

Some other solutions / thoughts?

Going back to a portion of your original suggestion, why not display a dummy image to the end users. Perhaps a default IH4 image (as opposed to the Zen Cart default product image) so that the shop owner CLEARLY knew what the issue is. (If they see the default image, this means an issue with their image upload has occurred. And of course, send an email to the Zen Cart administrative account to report to the admin user(s) the image needs to be replaced with a lower resolution copy (possibly with a calculated max resolution) or contact their hosting provider to increase the amount of memory available to a PHP script.

I personally would love to see the cart give real information - hey, dummy, that image is too large to display! Something really up front that tackles the real problem. This happens on images so large that I have trouble understanding why any cart owner would think it is an appropriate image. I don't want it to work on those images in reality. They should be fixed - only by alerting the cart owner to the problem can that be tackled.

But understand, every single time I've run into this issue so far has been when someone has an old cart with the old image handler and I upgrade the cart. Then the problem shows up long after - maybe even years after - the image was originally uploaded. So carts where they have been able to get away with the large images before. I have never had someone bring this problem to me with a new cart or right after the installation of image handler.

delia:

I personally would love to see the cart give real information - hey, dummy, that image is too large to display! Something really up front that tackles the real problem. This happens on images so large that I have trouble understanding why any cart owner would think it is an appropriate image. I don't want it to work on those images in reality. They should be fixed - only by alerting the cart owner to the problem can that be tackled.

But understand, every single time I've run into this issue so far has been when someone has an old cart with the old image handler and I upgrade the cart. Then the problem shows up long after - maybe even years after - the image was originally uploaded. So carts where they have been able to get away with the large images before. I have never had someone bring this problem to me with a new cart or right after the installation of image handler.

Right.. the issue has nothing to do with Image Handler.. Frankly Image Handler hasn't really changed enough over the years that any one change would have concealed or created this issue.. The sum total of changes between IH2 and IH4 are as follows:

• Updates to support PHP version updates
• removing safe mode options
• addition of the auto installer
• the removal of no longer needed features
• proper setting of folders permissions (changed from 777 to 755)
• javascript updates to support browser changes

This has ALWAYS been a server side out of memory issue.. Because the image processing of these large images consumes more memory that allocated, GD cannot process the image on the fly to generate the optimized product images when displaying the product page (hence the blank product page). It looks like the error message output was disabled in IH4 so no one ever saw any specific error message..

So the "fix" as it were in summary is to add functionality to IH4 that basically slaps a shopowner on the hand at the time one of these images are uploaded. For images uploaded using FTP or a shop which is installing IH4 for the 1st time and may have existing images which are too large to process, IH4 will display a default IH4 specific image on the front end when image optimization processing will exceed the memory allocation..

Yes, I like that!

delia:

Yes, I like that!

Me too.. wish I could code it.. :laugh:

So for now the solution will be some enhanced user instructions.. DO NOT UPLOAD 5000x5000 pixel images!!!! :laugh:

5000x5000? They are asking for a :bangin:
:cheers: Or at least some good educational materials in the documentation :)

Wish I had the time to code the changes (and lots of other thoughts / ideas). But still working on some other pressing projects.... And for Zen Cart, some "plugin manager" related code... The current adventure: some semi-automated methods of handling binary files changes (so backing up, replacing, and restoring) during installation, upgrades, or removal of a plugin...

But at least there is now a reference / discussion on here for others (or myself when I find some time).

lhungil:

5000x5000? They are asking for a :bangin:
:cheers: Or at least some good educational materials in the documentation :)
Yep!! and documentation I can do.. I'm also gonna take a swing at making the option to process images using ImageMagik something that be turned on in the admin (THIS is something I think I can execute on my own..)

lhungil:

Wish I had the time to code the changes (and lots of other thoughts / ideas). But still working on some other pressing projects...and you KNOW I wish you did too!! :laugh: IH4 could use yourr or lat9's golden hands!!:laugh:

lhungil:

But at least there is now a reference / discussion on here for others (or myself when I find some time).Yep.. Agreed.. :yes:
Funny how as LONG as I've been in the thick of IH4's inner workings, it seems like I still discover something NEW in Tim's brainchild..