Froogle Merchant Center merged into GoogleBase

BlessIsaacola:

Nope I did not clear it but googlefroogle works fine in either case as long as the max_execution_time is enough so I am cool! I just found another mod that needed the max_execution_time set to 300 or unlimited.

Good to hear you have found a solution to your timeouts and thanks for sharing. Can you report if you're using Andrew's latest mod version, which still includes label and currency fields in the generated feed and which are no longer "supported" by Google? And if so was your feed upload accepted and published by Google? And did you do bulk upload via GoogleBase web interface or did you FTP?
With the number of products you have what is the size of your feed?

Or did you manipulate Andrew's code to remove label and currency attributes?

Woody

So am I to assume everyone using the FroogleGoogle mod is doing the uploads manually or via the website? My file is generated fine, but I had to manually upload it to have it even register.

ecobunz:

So am I to assume everyone using the FroogleGoogle mod is doing the uploads manually or via the website? My file is generated fine, but I had to manually upload it to have it even register.

FTP works for me. Both manual FTP and FTP via cron job.

Woodymon:

Good to hear you have found a solution to your timeouts and thanks for sharing. Can you report if you're using Andrew's latest mod version, which still includes label and currency fields in the generated feed and which are no longer "supported" by Google? And if so was your feed upload accepted and published by Google? And did you do bulk upload via GoogleBase web interface or did you FTP?
With the number of products you have what is the size of your feed?

Or did you manipulate Andrew's code to remove label and currency attributes?

Woody

I am using Andrew's version 1.3.3 NOT 1.3.3a available in the download section by another author.

The Label and Currency are included in my generated file but that doesn't throw an error on Google side because neither of the above fields are part of the required fields. Google is more concern that the following required fields are present:

brand
condition
description
expiration_date
id
image_link
link
price
product_type
title

Because by default, our product type is other we bypass the requirement for other Product_Type specific requirements.

Our product feed was accepted and published when manually or automatically uploaded to Google.

The size of our file is 8.01MB fro ~10,000 products.

No manipulation has been done to Andrew's mod at this time. The only thing I temporarily turned off is Watermark for images which was making GoogleFroogle mod to generate new large image on the fly.

If there is anything else you'd like me to answer please let me know.

Isaac.

ecobunz:

So am I to assume everyone using the FroogleGoogle mod is doing the uploads manually or via the website? My file is generated fine, but I had to manually upload it to have it even register.

FTP through Zen Cart Admin, Cron and Web upload through Google interface all works fine for me and the feed is accepted and published.

BlessIsaacola:

If there is anything else you'd like me to answer please let me know.
Isaac-

Appreciate the detailed info. Certain to help others analyze feed issues.

I've had similar results except I don't have near that many products and was not seeing the IH2 water mark issues and timeout issues as you previously reported.

And good to know about the product_type exception. I did not know that assigning "other" for the product_type allowed for exceptions.

I've been treading through some recent GoogleBase docs and Google highly recommends five additional attributes over and beyond the old base Froogle feed and also have repeatedly reports that label and currency attributes are no longer "supported". But as both you and I have observed and reported Google appears to be continuing to accept/publish old-style froogle feeds and are not rejecting feeds containing the two now unsupported attributes and not including five extra recommended attributes. I am glad to see that as we were previously under the assumption that Google might start rejecting old froogle feed schema which would then cause Andrew's mod to be no longer useful, (unless some code edits were implemented).

BTW the only difference in the 1.33a version in the downloads area from Andrew's last version (1.33) is in the 133 updater SQL patch. The 1.33a uploaders intent was to repair an issue with the 1.33 version. But I found the updater SQL in 1.33a is problematic in itself.

It would be good if folks "fixing" mods developed by 'real developers' would allow others to test their code fixes/edits, before uploading to the downloads area. Q&A does not necessarily mean questions and answers.

Woody

I'm using the 1.3.3a version. It's strange, I can use the file from the addon and manually upload, yet the cron job version seems not to work for me. At least when I check the latest upload it's the manual version. Hrm wonder whats up.

BlessIsaacola:

FTP through Zen Cart Admin, Cron and Web upload through Google interface all works fine for me and the feed is accepted and published.

Woodymon:

Isaac-

Appreciate the detailed info. Certain to help others analyze feed issues.

I've had similar results except I don't have near that many products and was not seeing the IH2 water mark issues and timeout issues as you previously reported.

And good to know about the product_type exception. I did not know that assigning "other" for the product_type allowed for exceptions.

I've been treading through some recent GoogleBase docs and Google highly recommends five additional attributes over and beyond the old base Froogle feed and also have repeatedly reports that label and currency attributes are no longer "supported". But as both you and I have observed and reported Google appears to be continuing to accept/publish old-style froogle feeds and are not rejecting feeds containing the two now unsupported attributes and not including five extra recommended attributes. I am glad to see that as we were previously under the assumption that Google might start rejecting old froogle feed schema which would then cause Andrew's mod to be no longer useful, (unless some code edits were implemented).

BTW the only difference in the 1.33a version in the downloads area from Andrew's last version (1.33) is in the 133 updater SQL patch. The 1.33a uploaders intent was to repair an issue with the 1.33 version. But I found the updater SQL in 1.33a is problematic in itself.

It would be good if folks "fixing" mods developed by 'real developers' would allow others to test their code fixes/edits, before uploading to the downloads area. Q&A does not necessarily mean questions and answers.

Woody

Thanks Woody for the response. We are taking the approach that we will not chase every thing Google says (because Google talks too much :)) until it becomes necessary to change.

However, in light of our dialogue on this forum, I have disable the currency attribute on the Admin side. That reduced the size file a little bit as well. I am sure Google is going to be annoucing more changes with GoogleFroogle and hopefully, someone will pick off where Andrew left off to keep this great mod updated.

Isaac.

ecobunz:

I'm using the 1.3.3a version. It's strange, I can use the file from the addon and manually upload, yet the cron job version seems not to work for me. At least when I check the latest upload it's the manual version. Hrm wonder whats up.

Did you install 1.33a fresh or update over a previous GoogleFroogle mod version?

Test if you can manually FTP your feed from a FTP client either on your PC via a FTP client hosted on your server (not a web based upload).

Note the cron job doesn't actually FTP. It just pings Google to come and get it. After the ping, the come and get it part can take awhile.

Woody

BlessIsaacola:

We are taking the approach that we will not chase every thing Google says (because Google talks too much :)) until it becomes necessary to change.
This is prudent ;-)

I noted in the GoogleBase Blog a developer reported recently that another significant feed schema change would be announced/introduced shortly. I'm awaiting to hear details.

BlessIsaacola:

However, in light of our dialogue on this forum, I have disable the currency attribute on the Admin side. That reduced the size file a little bit as well. I am sure Google is going to be annoucing more changes with GoogleFroogle and hopefully, someone will pick off where Andrew left off to keep this great mod updated
In version 1.33 I don't see method of disabling currency in admin. I see a "default currency" dropdown with five choices but one must be chosen. Did you hack the code? Or did I miss a SQL patch somewhere along the way?

And whatever you did to "disable the currency attribute on the admin side" did that also remove the currency type attribute in the "link" URL parameters. e.g.

http://www.domain.com/shop/index.php?main_page=product_info&products_id=12&currency=USD	50.00

I'm thinking the price also does not need to be in the link URL?

I will take a stab and removing label attribute outpout from the feed generator code. Will test it out and the post if successful. Should be fairly straight forward, even for a non-coder type like me.

Yes an XML based RSS2/GoogleBase feed generator for Zen Cart with ability to define your own attributes would be a very cool thing.

Woody

Woody, I have a feeling I misunderstood you. I thought you were talking about the currency showing which I disable by setting Show Default Currency to false.

BlessIsaacola:

Woody, I have a feeling I misunderstood you. I thought you were talking about the currency showing which I disable by setting Show Default Currency to false.

Doh!. I missed seeing the admin config etnry just above "Default Currency" . And I just set to false and answered my question. Set to false removes both the price and the currency from the link URL as well as removes the "currency" attribute field from the feed.

Thanks for the tip.

I also observe the product "brand" attribute is empty throughout the feed.

On one of my sites I'm using the Short Description mod for an "author" field. Thinking I will map that product field to the brand attribute for froogle feed output. And add a switch in admin to disable.

Or I could map the ZC product manufacturer field to the brand attribute in feed. That probably makes more sense, but not for the shop I'm working on at the moment ;-)

Note you can use the Short Description field for any type of product descriptor (e.g. author, publisher, etc).

Regards to the ID field in the feed I was considering mapping it tothe Zen Cart product model number. But ID value has to be distinct in the feed and some shopowners apply same model number to different products (as they use zen cart product attributes do the differentiating). Thus assigning model number to ID field is not a good idea. Will let it remain mapped to product_id.

Woody

Woodymon:

Note the cron job doesn't actually FTP. It just pings Google to come and get it. After the ping, the come and get it part can take awhile.
I wasn't thinking straight when I wrote the above. Forgive me. That point applied to old googleFroogle feeder version.

What does the cron job report you receive in email say?

The feed uploaded via FTP is not promptly processed/published by Google. Did you give it a day or two then check?

Woody

when you say "they come and get it" surely it just toses up a URL for them to grab it from, correct?

Woodymon:

Did you install 1.33a fresh or update over a previous GoogleFroogle mod version?

Test if you can manually FTP your feed from a FTP client either on your PC via a FTP client hosted on your server (not a web based upload).

Note the cron job doesn't actually FTP. It just pings Google to come and get it. After the ping, the come and get it part can take awhile.

Woody

ecobunz:

when you say "they come and get it" surely it just toses up a URL for them to grab it from, correct?See above post.

Very strange, I can see the files in /feeds on my server. I can manually FTP it. I can upload via the web page. I see the output in my cronjobs in my log files - but the googlebase uploads list never shows the bulk updates.

Will keep on troubleshooting.

ecobunz:

Very strange, I can see the files in /feeds on my server. I can manually FTP it. I can upload via the web page. I see the output in my cronjobs in my log files - but the googlebase uploads list never shows the bulk updates.

Will keep on troubleshooting.

Ecobunz: Just curious, is there ANY chance that your website is set up with/without "www" in front of it, and you set up google/froogle the OPPOSITE way?

This happened to me with the sitemaps, and once I figured that out, I got it all working.

I had set up a site in sitemaps as "www.mysite.com", but my installation of Zen Cart was "mysite.com". I THINK I had a similar problem with Froogle. I changed them all to match my site, and voila, they worked.

I can't seem to get around this time out issue. I hope this is something that can be fixed in the next version. I would be willing to pay a_berezin for a fix at this point.

I checked my error log via Cpanel and I am getting the following errors:

[Sat Mar 31 15:41:19 2007] [error] PHP Notice: Undefined variable: in /home/<username>/public_html/googlefroogle.php on line 278
[Sat Mar 31 15:41:19 2007] [error] PHP Notice: Undefined variable: retval in /home/<username>/public_html/googlefroogle.php on line 278
[Sat Mar 31 15:41:19 2007] [error] PHP Notice: Undefined variable: retval in /home/<username>/public_html/googlefroogle.php on line 281
[Sat Mar 31 15:41:19 2007] [error] PHP Notice: Undefined variable: retval in /home/<username>/public_html/googlefroogle.php on line 280
[Sat Mar 31 15:41:19 2007] [error] PHP Notice: Undefined variable: retval in /home/<username>/public_html/googlefroogle.php on line 278

As I said in PM, you're seeing those PHP notices because I (temporarily) turned on error reporting in your googlefroogle.php file to see if there was any infomation that would help with the timeout problem you said you were having. Error reporting is turned off now, and your googlefroogle.php file returned to its original state, so you won't see those anymore.

In case it might help anyone else, I'll describe briefly what appears to be happening. This is pretty well what I've already said to you via PM.

The problem you appear to have is not a timeout (the mod does loop through all the records in your database and reports the correct number), rather that only the first batch of records processed by the mod (c. 177 in your case) is written to the feed file.

After the first batch (in $output_buffer) is written to the feed file, $retval appears to become NULL in the function zen_froogle_fwrite() and further batches are not written. Each further attempt to write the contents of $output_buffer to the feed file appears to result in an 'undefined variable' notice

If you count the number of 'undefined variable' notices that appear (after you've run the feed generation) and multiply them by the size of the first batch written (c. 177 records) it comes out pretty near to the number of records in your database.

I tried this on a shop of mine where the feed does complete successfully (ie. all records are processed and written), and (with error reporting turned on) the same PHP notices appeared, so the appearance of the warning notices doesn't seem directly related to whether all the batches of records are written or not.

Searching on php.net, it appears there is/was a bug in PHP 4.3.2 that can result in the returned variable from a function becoming (or being reported as) NULL after the first iteration, although it appears that this doesn't always affect the output of the variable - ie. sometimes the variable still functions, but sometimes it doesn't - put another way, it appears the behaviour can be unpredictable.

It's not clear from the bug report(s) whether this bug has been fixed in PHP 4.x.x or only in PHP 5.x.x.

http://bugs.php.net/bug.php?id=22836%22

But whether this is the cause of your problem or not I don't know - perhaps someone else can chip in with their knowledge/experiences.

Did you try installing a stock zen demo shop in a subdirectory on your server with just the Froogle mod installed as I suggested? If the Froogle mod runs and completes all records on a stock zen demo shop, you will know it's not your server setup.

Then load a copy of your live database into the stock zen demo shop and try running the Froogle mod again - does it complete all records? - if it does, then you know it's possibly another mod you have installed on your (live) shop (or a difference in admin settings perhaps) that's causing the problem.

If it doesn't complete all records, then it's possibly something in your live database that's causing the problem.

In any case, it should help you to narrow things down a bit.

As I said in PM, please make sure you change your passwords on cpanel, FTP and your admin now so you can be sure I no longer have access - this is for my security too. :smile: