Odd link problem after upgrade

I'm not sure if this started immediately after a recent upgrade to 1.3.9h or later, but we just noticed it.

On this site, if you scroll down to the bottom of the home page to the New Products For May box, and hover over any of the products, you'll see a slight difference in the text links compared to the image links. For example...

text link:  ../index.php?main_page=product_info&cPath=130&products_id=964
image link: ../index.php?main_page=product_info&cPath=130&products_id=964

Just before the products id in the text link (but not in the image link) the normal ampersand is being changed into an HTML entity and then the link doesn't work. Oddly, the ampersand for the category is not being changed.

If you check out any other listing, for a category for example, or even by following the New Products... link under the categories, all the links look and work correctly. It's only wherever you have a New Products For May box that the links are wonky.

So I guess my question is... where is the code that is outputting those links? I'll need to check that php file against a stock one to see what's wrong, but I don't know where to start.

Thanks for any hints you can offer.

Rob

rstevenson:

So I guess my question is... where is the code that is outputting those links? I'll need to check that php file against a stock one to see what's wrong, but I don't know where to start.

/includes/modules/new_products.php

Not sure if that'll be the offending file, but it appears to be a good place to start.

Cheers
Rod

ps. Followed this further... also check /includes/init_includes/init_canonical.php

Thanks Rod. The init_canonical.php file is identical to a stock one. However, the new_products.php file is part of the Single Listing Template add-on, which also installed a modified version of includes/templates/MY_TEMPLATE/templates/tpl_products_new_default.php. I could try removing that mod, but would prefer to run it, of course. It doesn't seem to have a switch in admin to turn it off for testing, but it does have a debug function. I'll turn that on and have a look.

Rob

If it were me, I'd just rename includes/templates/MY_TEMPLATE/templates/tpl_products_new_default.php which will cause zen to use the original template. It'll probably screw up the look of the page, but it'll prove or elimate this file as being the culprit. (I call this the 'fck all to infinity' check). :)

Cheers
Rod

Nope, no debug function. The readme says to "enable the Admin->Configuration->Index Listing - Template Debugging = True and see what files are being used." Unfortunately there is no such setting. Something is amiss. Must go check the files now. [sigh]

But I like your 'fck all to infinity' check. I'll try that too. >>>

Tried that and the New Products... listing returned to one product per row, as expected. But the New Products For May box didn't change and the links remained wonky. But that's an interesting technique. I'll have to remember that one.

Rob

rstevenson:

But I like your 'fck all to infinity' check. I'll try that too.

That was taught to ne by my electronics teacher several decades ago. Initially as a way to determine what would happen at point 'A' of a circuit if resistor 'B' was increased or decreased in value. Instead of trying to figure the answer out using minor changes, the idea was to figure it out as if the resistor had a dead short (fck all resistance, and again as if the resister was open circuit (infinite resistance). These 'figures' made it easy to see what the answer would be. Any values in between those two was just a matter of degree.

This same princple works with many other real world situations as well. :)

Anyway, I digress...

Another place for you to check would be

/includes/functions/functions_categories.php
This may be counter intuitive but 'new products' appear to be processed just like any other category.

Incidently, when you've fixed this problem, I think it about time you updated Johanne's shipping module. <smiles> I see she's still using one of the V2.x.x series.
The V3.x.x series is an easy upgrade (just copy over the new files, and navigate to the admin settings). The act of navigating to the admin settings will automatically update the database files whilst keeping most, if not all, of her current settings intact).
One of the biggest benefits she will gain is a performance boost and increased reliabilty. V2 only 'knows' about one of our three servers. V3 knows about them all.

Cheers
Rod

Another place for you to check would be
/includes/functions/functions_categories.php

Nope, that one's stock too.

Rob

Just done a bit more investigationing, and have found;

/includes/modules/new_products.php is the only file that has a reference to class=centerBoxContentsNew, which if you view the offending page source is the class id used in the faulty output.

Furthermore, line#72 of this file (V1.3.9 distro) has the following code:

'text' => (($new_products->fields['products_image'] == '' and PRODUCTS_IMAGE_NO_IMAGE_STATUS == 0) ? '' : '<a href="' . zen_href_link(zen_get_info_page($new_products->fields['products_id']), 'cPath=' . $productsInCategory[$new_products->fields['products_id']] . '&products_id=' . $new_products->fields['products_id']) . '">' . zen_image(DIR_WS_IMAGES . $new_products->fields['products_image'], $new_products->fields['products_name'], IMAGE_PRODUCT_NEW_WIDTH, IMAGE_PRODUCT_NEW_HEIGHT) . '</a><br />') . '<a href="' . zen_href_link(zen_get_info_page($new_products->fields['products_id']), 'cPath=' . $productsInCategory[$new_products->fields['products_id']] . '&products_id=' . $new_products->fields['products_id']) . '">' . $new_products->fields['products_name'] . '</a><br />' . $products_price);

Take special note of the ampersands immediately preceeding the "products_id=". I'm wondering if these are somehow related to the problem. Using my fck all to infinity technique, try removing them and see what effects (if any) it has on the offending page. I'd also go as far to suggest that you change "products_id" to something like "PRODUCTS_ID".
Yes, this WILL break things big time, but all being well, when you view the source (or hover over the links) you'll see quite clearly if we are on the right path or not. If nothing changes, put things back and we'll need to start looking elsewhere. If you see the changes reflected on the page we'll know that we are at least getting a little closer to the problem

Good luck... I'm about to call it a night (1am here, again).

Cheers
Rod

Thanks Rod. I'll give that a try, though as I mentioned before, that file is stock right out of the clean untouched copy of ZC 1.3.9h I always keep handy for times such as this.

Rob

rstevenson:

Thanks Rod. I'll give that a try, though as I mentioned before, that file is stock right out of the clean untouched copy of ZC 1.3.9h I always keep handy for times such as this.

Rob

Didn't want to mislead you. I wasn't stating that this file was the one causing the problem, it is just a 'convenient' one to start debugging with :)

Anyway, I've done those tests (and more) on my own system, and can confirm that this is indeed the page where the links are being created, BUT, somewhere between here and the page output appears to be having a bit of sanitation applied, and it appears that this code (whereever it is) is the culprit.

What I found (FYI) is that removing the ampersand breaks things (as I expected), BUT, if I add several of them (eg "&&&&&product_id") all but one of them get stripped out. No sign of them being converted to &amps; though. For further testing I replaced the first one with "&&&&X&&&&products_id" and the other with "&&&&Y&&&&products_id" -- and the output was the same type of sanitation... IOW, the output became "&X&products_id" & "&Y&products_id"

So, where to go from here.... for me, it's definelty bed :)

For you.... Hard to say... From this point on, I would be loading up my IDE (netbeans), adding a break on this line, then literally stepping through the code to see where this 'sanitation' is taking place and where it is getting htmlalised.

It is not much point in my doing this with my test site though, because mine doesn't exhibit the same problem as yours (I've tried both V1.3.9 and V1.5.0).

In short, I think I've now taken this as far as I can with you.... UNLESS perhaps you find if you make the same edits as I did and get entirely different results...

Cheers
Rod

G'day Rob,

Methinks I'm getting closer.

Check your file
/includes/functions/html_output.php

Specifically look for the lines that follow

// clean up the link before processing (line#72) and
// clean up the link after processing (line#91)

This is where the URL strings are cleaned up.

Also, double check that this file is the correct one.
There is another similar file, also called html_output.php in the /admin/includes/function/ folder.

I image it would be quite easy to edit the wrong one of these two files if not careful (much like the two configure.php files).

Although not pretty, if you find the line

return $link ; (line #96) then just befor that add

echo $link ;

You'll see all the links being dumped to screen (when you visit the site) with or without the errors.
Also, in the lines before that you'll see several places where the ampersands get manipulated/processed. If you comment out these one by one (with the echo line still in place) you will be able to quickly narrow down exactly which one of these is causing the problem... That assuming you need to continue to fault find rather than replacing the offending file.

Back in your hands now :)

Cheers
Rod

Thanks Rod. You poked me in the right direction.

I checked the /includes/functions/html_output.php file against a clean copy from my untouched 1.3.9h files and it is, as I suspected, just the default one. I also checked all mods installed, and the few that added anything to the includes/functions folder did so correctly, by using the extra_functions folder, and none of them modified that particular file anyway. But...

Looking at that file, just above where you said to insert the echo line (which I tried, but no lightbulbs went off over my head), I saw this line...

$link = preg_replace('/&/', '&', $link);

I'm not much of a hand at PHP but that seemed to me to be replacing an escaped ampersand with the offending HTML entity ampersand. So I commented that line out and reloaded the page. Et voila! The problem disappeared, all links seem to be written correctly and all work correctly.

I think it's safe to leave everything like this, since there is clearly another line above that one which is concatenating any extra ampersands properly. I'll test all the way through an order just to be sure. But now the question becomes, why would I have to do that to a default ZC file which works fine as is on other sites? Or maybe, why was the New Products For [Month] box bypassing that clean up routine when it too is standard ZC? Or maybe the real question is, how many lines of code does Zen Cart contain? :shocking:

Rob

rstevenson:

Looking at that file, just above where you said to insert the echo line (which I tried, but no lightbulbs went off over my head), I saw this line...

$link = preg_replace('/&/', '&', $link);

I was wondering about that too. Hense my suggestion to comment out such lines one by one until you find the 'right' one. :)

rstevenson:

I'm not much of a hand at PHP but that seemed to me to be replacing an escaped ampersand with the offending HTML entity ampersand. So I commented that line out and reloaded the page. Et voila! The problem disappeared, all links seem to be written correctly and all work correctly.

I think it's safe to leave everything like this,

I'd like to seek advice from the dev team before I'd be assuming this is 'safe'.

rstevenson:

since there is clearly another line above that one which is concatenating any extra ampersands properly. I'll test all the way through an order just to be sure. But now the question becomes, why would I have to do that to a default ZC file which works fine as is on other sites? Or maybe, why was the New Products For [Month] box bypassing that clean up routine when it too is standard ZC?

These are questions I also asked myself, which is why my gut feeling is that this 'fix' is actually hiding the real source of the problem (which I've not been able to duplicate). Hiding the symptom without knowing the cause is risky business. It would be interesting to know/see what the $link variable looks like as it entered this function. It probably contains some kind of weird character or formatting that isn't being correctly processed during the pre-processing, which in turn would indicate the additional code is needed to take care of such situations. The post processing (the line you have commented out) is/was actually doing what it was supposed to be doing in 'trapping' such errors before they could cause any damage.

Input from one of the dev team is probably essential at this point.

Cheers
Rod

I've done a rather cursory quick-read of this thread.
What I'm not clear about is the origin of the problem.
Does the problem manifest when using only core ZC files? Or only when using a certain addon?

Well, the core files work fine of course, so it has to be one of the add-ons. But the problem exists in the New Products For May section on any page that displays that, but doesn't manifest itself in any other link, not even the New products... links.

What I'd better do is un-comment that "fix" and then switch to the Classic template just to make sure. Just a sec ... ... .

As expected, the Classic template outputs the New Products For May links just fine. So it's coming from a mod. Now how do I find out which one? Most of the mods installed would have nothing to do with outputting links. Here's a list of them in case you can spot a likely culprit...

Admin Login As Customer 2.2
Blank_Sidebox_2.0
categories_dressing_2-7-3
column_divider_pro_cdp_v0-4
credit_card_by_phone_or_fax_ccbyfax_1-8b
currency_dropdown_header
Direct Bank Deposit v1.1
EasyPopulate-4.0-9773477 (just for downloads)
Geotrust
HoverBox
LiveHelp
netregistry
ozpost_V2-0-5
reward_point_full_suite_1-33a
reward_points_page 1.0
search_log_2-1
single_listing_template_1-8

Rob

rstevenson:

As expected, the Classic template outputs the New Products For May links just fine. So it's coming from a mod. Now how do I find out which one?

Disable them one by one until the problem goes away :)

rstevenson:

Most of the mods installed would have nothing to do with outputting links. Here's a list of them in case you can spot a likely culprit...

If I had to take a punt on where to start, I'd go for:

1. categories_dressing_2-7-3
2. column_divider_pro_cdp_v0-4

I've only included 2) because I know nothing about this one.

Categories dressing is my first choice for 2 reasons. 1) It was developed for Zen 1.3.8 and hasn't had an update since 2009, which makes it pretty ancient, and 2) it is the only one in the list that appears to have anything to do with the category links, and since 'new products' is essentially just another category type it wouldn't be hard to imagine how this could get things a little wrong.

This really is nothing more than a stab in the dark though.

Cheers
Rod

RodG:

... and 2) it is the only one in the list that appears to have anything to do with the category links, and since 'new products' is essentially just another category type it wouldn't be hard to imagine how this could get things a little wrong.
Just want to make sure we're not talking at cross purposes here. "New products...", the link down at the bottom of the left column, below the Categories links, goes to a page on which all the links work fine. The problem is showing up only in the New Products For May box, on any page which shows that box.

I'll check with the site owner about disabling the mods one by one. But I'll try those two first.

Rob

Okay, we can eliminate the Categories Dressing mod from consideration. It was working in a previous template but is not used in this one. Also I tested the Column Divider Pro mod and removing it had no effect on the problem.

Rob

Two down, sixteen to go... Hmm, make that 15... you can eliminate ozpost. :)

These can probably be rules out too.

credit_card_by_phone_or_fax_ccbyfax_1-8b
currency_dropdown_header
Direct Bank Deposit v1.1
EasyPopulate-4.0-9773477 (just for downloads)

Cheers
Rod

ps. Don't forget to clear your cache after removing/changing ANY of the modules. (Shift F5 is usually enough). I've lost count of the number of times I've changed something and seen no immediate difference, and gone around and around in circles as a result. :(

I think I found it. In the Single Listing Template mod, in the file includes/modules/TEMPLATE/new_products.php the bit of code that collects up the product id is this...

'cPath=' . $productsInCategory[$new_products->fields['products_id']] . '&products_id=' . $new_products->fields['products_id']

But in the file includes/functions/extra_functions/index_listing.php, it's this...

'cPath=' . $productsInCategory[$products->fields['products_id']] . '&products_id=' . $products->fields['products_id']

Note the ampersand as an entity instead of just an ampersand.

Now to change that and see what happens... ... ... . Bingo! That fixed the problem -- correctly, this time, I think.

Thanks Rod for the pushes in the right direction.

Rob