html_header and noindex,nofollow

[Pauls]

Hi,
i am struggling with one of my sites using version 1.3.7 getting the googlebot to visit my site. Obviously i considered many of the things I do to try and generate good ranking could be a reason why googlebot hasn't visited my site since March 2007! I made lots of changes etc but nothing seemed to work to bring the googlebot back.
The i looked at the source code to my index page, and found meta code tags telling the bots to go away. This was the exact code being generated dynamically.

<meta name="robots" content="noindex, nofollow" />

Looking at includes/templates/yourtemplate/common/html_header.php I noticed included the following code

<?php if (defined('ROBOTS_PAGES_TO_SKIP') && in_array($current_page_base,explode(",",constant('ROBOTS_PAGES_TO_SKIP'))) || $current_page_base=='down_for_maintenance') { ?>
<meta name="robots" content="noindex, nofollow" />
<?php } ?>

Now please excuse my ignorance but isn't that stating to all bots don't follow or index this page? I realise that it should be a proviso of if the page is the down for maintenance, however when I removed this code there was no longer the noindex nofollow code within my normal web pages as shown previously.

If that is so then no search engine will index my sites page, surely this is not proper? Am I missing something, as this suggests to me that my pages that are generated using html_header are telling the search engine bots to go away!!! The last thing I (or anyone else would)want to happen!

Thanks for any help.

[birdoasis]

You don't want every page followed because those pages will be considered duplicates. So the dev's have put the noindex, nofollow in the pages that are on ever single page.

For instance, you defiantly don't want hundreds of instances of the header to be seen by the spider.

If you look deeper at the code you'll find an include statement with the metatags.php page or something similar.

That way the search engines will pickup just the pages with content and product on them. This helps with search engine optimization.

Now I haven't actually submitted my site but I've got goolge spidering it's way thought my site. The only reason's I can think of is that I've put my products into google base, and I'm using google checkout.

Hope this helps.

[Kim]

Paul-

What is the URL of your site?

[tonibarcelona]

is there any changes at v. 1.3.7 versus 1.3.6. at managing of tags noindex and no follow. It blocks the same pages than at 1.3.6?

[tweakservers]

I am having the same problems here. Submitted the site to Google Sitemap but nothing shows up. Is the NoIndex, NoFollow the main cause of this ? My site is tweakservers.com

[Kim]

You source code says index, follow.