Re coding Site map feature to include all products

Hi,

I am wondering what kind of code it would take to change the site map display that is currently generated by zencart 1.5.4 to allow it to show all the products in the catagories as well as what it shows now.

I only have a few and would like them shown AND will use the site map link on my information site as well.

One reason I want this is that I not sure my products are getting indexed by google etc. Having the site map both in main directory for the bots and on the information site may help. I am ignorant but there is some logic in this :)

Mike

That kind of depends on what that logic is based. Current "indication" is some perceived issue of the site being indexed. Has a webmaster tool been installed to evaluate the ability of and success of being indexed?

As far as a sitemap for the bots, have you installed something like sitemapxml to reduce the number of files attempted to be searched and simplify the process?

Have you added a robots.txt file that is preventing navigation of your site by indexers?

Lastly (though perhaps most important if gaging index problem against the site being found in an internet search), do you have textual content that sets your product out for those that are looking for your product?

Unrelated to the above issue, why is ZC not your linformational" site? Ask because what money is your informational site/section making and how far away from that does a customer have to go to make such a purchase?

Thanks for the reply.

Ok i have 21K hits to a 403 error page. I also have 16k hits other than the 21K error hits.
I deleted anything that could be causing this. I am suspecting it could be an ajax problem but that is just an uneducated idea from me.

Has a webmaster tool been installed to evaluate the ability of and success of being indexed?

I have not installed any tool that can verify indexing.

As far as a sitemap for the bots, have you installed something like sitemapxml to reduce the number of files attempted to be searched and simplify the process?

No, From what I saw of the finished page it looked not much different that the site map already made by zencart 1.5.4

Have you added a robots.txt file that is preventing navigation of your site by indexers?

I do not know which folders to allow them into. I did the following:
User-agent: ia_archiver
allow:
User-agent: Alexabot
allow:
User-agent: verifybot
allow:
User-agent: *
Crawl-delay: 86400
Disallow: /assets/
Disallow: /awstats/
Disallow: /cgi-bin/
Disallow: /awstats-7.4/
Disallow: /counter/
Disallow: /modlogan/
Disallow: /webalizer/
Disallow: /zen-cart6/xxxxxxxxx/ xxxxx being my admin area

I am also having trouble getting Alexa to index site. I did all they asked and still get error. My htaccess file there does not exclude them according to my server tech support.

Unrelated to the above issue, why is ZC not your linformational" site? Ask because what money is your informational site/section making and how far away from that does a customer have to go to make such a purchase?

I am not a coder but I built the site and the zencart though I did have someone do the responsive on the site.
http://ultimatepicklejar.com/ is the information site and the store is from a shopping link in head menu from all the info pages to the store:
http://www.ultimatepicklejar.com/zen-cart6/
Also I am going to change the way the address to the store is seen later to reflect something other than "zen-cart6" ......

It would be more handy to have the one site for sure. I could see it working if i knew how to add to the menu bar at the store and I could probably fix a blank sheet to add the text pages too. And then I could easily move the whole store to the main root area where the store is now..... Good idea.

EDITED THIS: Information page is pretty important as product is a new one and needs describing if it is to catch on.

All this does not much matter as we have not been able to get people to come to the store but we do ok on Ebay and Amazon .

Mike

Change your admin directory NOW. Then remove it from the robots file...explain more in a moment.

The robots.txt file is a plain text file accessible by anyone that can browse to your site. Good bots. Will listen to what is in there, bad bots or now people visiting this post can obtain the admin address by "reading" all that they need in order to get to it...

The file is intended to advise against indexing/navigating the pages in an open suggestion format. Not to literally prevent access of the information at those directories.

The whole file or just from the robot txrt???????

good thing then I just had that up for a day messing with Alexa
! thanks

ParseRaider:

The whole file or just from the robot txrt???????

I haven't evaluated the remaining content, only saw the identification of the content in the robots.txt file and then the full path to the store.

So first, rename admin directory, second, remove the reference to the admin directory in the robots.txt file.

If the install was done correctly that is all that is needed to "move" the admin section to a new place. Otherwise, may have to modify the admin/includes/configure.php file to identify the new directory path for the admin.

Man thanks!

DO i take out Disallow zen-cart6/ of leave it?

I feel you saved my life thanks!

I checked configure.php and is not reference to old admin area name.... Whew!

ParseRaider:

Man thanks!

DO i take out Disallow zen-cart6/ of leave it?

I feel you saved my life thanks!

So that kind of depends on your intent with all that you have in your robots.txt file...ne of the issues described is indexing... So, if the folder related to your site is set to be "hidden", then certainly not helping matters... But, also the arrangement of search engines "should" play a factor in that... Supposedly if an agent is identified, then that agent will only pay attention to the information in it's section and ignore the others... Including the * agent... But, they could also try to process their section and the all others section. That aspect also brings to question the intent of such a large crawl delay... One page attempt every 86400 seconds? If you only want to be indexed by the few identified agents, okay, but I doubt that is the desire...

This also could be one reason certain bots haven't been able to gather information to appear in an index list... Once a day, allowed to visit the site (again just a suggestion), and every second of delay means the next visit is that much longer away....

Okay, sitemapxml generates a complete list of "interesting" pages in a single start and in a file recognized by many search engines. Therefore, they look for that file, find it, then go from there and the subsequent page(s). Otherwise they take a guess off of the page on which they land, and work only from there. Even if they find the ZC sitemap, there are a lot of other pages related...

Now the other side is to have all the files of the site mapped to a sitemap file, but then there are files identified that the search engine can't actually get to... Try loading includes/application_top.php from your browser... Don't work do it? :)

I'm sorry, but I need to step away for a bit. Have some things to do. Hopefully someone else can come along and help further....

You saved my life :) I will wait until I have money to hire someone to fix the whole thing.

So you said you had a bunch of 403 errors. Indications as to which directories/files?

Recommend at least checking out google webmaster tools and setting that up to allow monitoring/inspection of site activity/problems...

I had a temp store that may have been active. I cant tell from awstats just where the 21K 403 hits came from. But I went over everything else and made sure all links are right. Killed the temp store and will see if it stops them. That is like 60%+ of the traffic on the site :(

I was able to set it all up and can run it but I think the need for some tech help is on the menu.

Thanks for the help. I also see the meta on the pages is not very good as the description is not showing keywords seperated by commas but all ran together. That would hurt ranking and search but would not have anything to do with the 21K error pages.

ParseRaider:

I had a temp store that may have been active. I cant tell from awstats just where the 21K 403 hits came from. But I went over everything else and made sure all links are right. Killed the temp store and will see if it stops them. That is like 60%+ of the traffic on the site :(

I was able to set it all up and can run it but I think the need for some tech help is on the menu.

Thanks for the help. I also see the meta on the pages is not very good as the description is not showing keywords seperated by commas but all ran together. That would hurt ranking and search but would not have anything to do with the 21K error pages.

I half expected another zenner to post about the above... I wouldn't focus too much on making/applying metatag information. It could also be a negative factor for some search engines. There's plenty of information/discussion through the forum about this aspect. I really would suggest getting like a google webmaster's account and set it up for your site. It will provide (over time) information related to the accessibility of the pages and particular issues that probably should be addressed. It is also important to understand the source of the traffic... Not good to find out that the thousands of site hits are actually coming from some automated system that has found a flaw with the site and is taking advantage of it for whatever reason is seen "fit".