SitemapXML v.2

[mc12345678]

Thank you mc,
Sorry, I meant all the .xml files associated with the sitemap plugin and the .htaccess in the sitemap folder.

I previously had mentioned two different values when discussing files and folders. Files tend to be 644 and folders 755. I'm not sure what drove the discussion of folder permissions to then apply those same values to files especially after suggesting to identify likely values based off of comparison of similar objects.

I don't know how much of the above is still an issue. I was reading the recent history and realized that file permissions may be set wrong as they were being modified to the settings discussed for folders. For files, would suggest the same type of review though would sample files within folders. For example there is an htaccess file within at least the first level folder of the zen cart install. Then there are other files within most folders.

[rlexyd]

Sorry mc, misunderstood you.
I've changed the files back to 644 and made the folders 755 which matches other files/folders, will see how Google sees them and go from there.

[rlexyd]

Still having problems, Google now seems to only fetch 1 of the sitemaps?!
Also, Google has crawled my sire but only main page and another, no other pages show up. Very weird.
I have Google AdWords which used to help index my site quickly but something not right this time round.

[rlexyd]

Had to change all the permissions for the generated .xml and .htaccess files associated with the Sitemap plugin to 755 as that seems to be the only way Google can fetch them all?!

[carlwhat]

Had to change all the permissions for the generated .xml and .htaccess files associated with the Sitemap plugin to 755 as that seems to be the only way Google can fetch them all?!

understanding linux permissions/security can be a bit of work. but it is wrong to say that this is the only way to do something.

why the permissions on your .htaccess file need to be changed, well that's probably not true. if you can go to:

www.yoursite.com/.htaccess

and you see your .htaccess file on your browser window, i would suggest to you that is a problem; and not how one would want their webserver configured.

and your sitemap program is NOT generating your .htaccess. if it is, there is something wrong with this picture.

if you can go to:

www.mysite.com/sitemapcategories.xml (or whatever your sitemap is called)

and you can see that xml file in your browser window, then so can google.

having to manually change your sitemap permissions every time you regenerate your sitemap is far from ideal, and something that i would bring up with your hosting company.

understanding the owner of the file, the group of the file, and which user is running apache (or whatever is serving up your web pages), is a starting point for being able to add a crontab entry for your sitemap, generating new sitemaps on a scheduled basis (based on how often your product mix changes/goes out of stock, etc.), and having those sitemaps be available to whatever spider wants to look at them, without the need to change permissions every time the crontab runs.

why they would need to be more than 644 is more about the setup than anything else.

[rlexyd]

Thanks for your assistance.
The sitemapxml plugin has a .htaccess file included in a folder called sitemap where the generated sitemap files are.
www.mysite.com.au/sitemap/.htaccess
However, when you go the the url to access the .htaccess file it doesn't load, it shows a permission error.
(It appears you don't have permission to access this page. 403 Error. Forbidden.)
Only the sitemap files load via the url, which as you advise, Google should be able to see them if I can see them, but for some reason, when I add the sitemap link to google search console and it accepts, but when I check it a day or so later, it shows some of the sitemaps as (couldn't fetch) which is so weird, and for some reason google/bing are only showing 2/3 pages as indexed.
I thought especially with AdWords it would start showing a few more pages..

[carlwhat]

Thanks for your assistance.
The sitemapxml plugin has a .htaccess file included in a folder called sitemap where the generated sitemap files are.
www.mysite.com.au/sitemap/.htaccess
However, when you go the the url to access the .htaccess file it doesn't load, it shows a permission error.
(It appears you don't have permission to access this page. 403 Error. Forbidden.)
Only the sitemap files load via the url, which as you advise, Google should be able to see them if I can see them, but for some reason, when I add the sitemap link to google search console and it accepts, but when I check it a day or so later, it shows some of the sitemaps as (couldn't fetch) which is so weird, and for some reason google/bing are only showing 2/3 pages as indexed.
I thought especially with AdWords it would start showing a few more pages..

you bring up numerous issues. and you tend to post things that others may read as being true which are not.

• 755 is not what i would recommend for your xml files.
• a htaccess file is not generated by this sitemap plugin. there looks to be a .htaccess file included and the author wants it placed in the directory where your sitemap files are. but very different from it being generated by a script. cpanel could generate one, but not this plugin.
• if the only files in your directory are xml files that you want to be seen by anyone browsing your site, the .htaccess file is not needed.
• i would suggest reading up on it: https://ithemes.com/what-is-the-htaccess-file/
• if you can see the sitemap xml files in your browser, so can google or any other search engine.
• why a page gets indexed (or not) is a separate issue and part of the black arts of SEO.
• what adwords has to be with anything, you got me. but marketing and SEO are not my thing. ensuring that your sitemap is in the correct format and readable is a different thing.
• if all of your sitemap files are correct, then the problem would be somewhere else.
• in general, i would guess that there is a problem with your sitemap files. and i would look at the ones that are not being indexed to ensure they look correct or at least like the other ones that are indexed.
• if they are correct and google does not index them, i would investigate with google. especially since you are paying them for adwords.

hope that helps.

[rlexyd]

you bring up numerous issues. and you tend to post things that others may read as being true which are not.

• 755 is not what i would recommend for your xml files.
• a htaccess file is not generated by this sitemap plugin. there looks to be a .htaccess file included and the author wants it placed in the directory where your sitemap files are. but very different from it being generated by a script. cpanel could generate one, but not this plugin.
• if the only files in your directory are xml files that you want to be seen by anyone browsing your site, the .htaccess file is not needed.
• i would suggest reading up on it: https://ithemes.com/what-is-the-htaccess-file/
• if you can see the sitemap xml files in your browser, so can google or any other search engine.
• why a page gets indexed (or not) is a separate issue and part of the black arts of SEO.
• what adwords has to be with anything, you got me. but marketing and SEO are not my thing. ensuring that your sitemap is in the correct format and readable is a different thing.
• if all of your sitemap files are correct, then the problem would be somewhere else.
• in general, i would guess that there is a problem with your sitemap files. and i would look at the ones that are not being indexed to ensure they look correct or at least like the other ones that are indexed.
• if they are correct and google does not index them, i would investigate with google. especially since you are paying them for adwords.

hope that helps.

Thanks for your reply :)
Yes, your are correct, the .htaccess file is included in this plugin and sits inside the sitemap folder that has the .xml files the plugin generates in admin.
As for the sitemap file, I ended up having to generate one from this link www.xml-sitemaps.com and use that instead and removed this plugin's generated .xml files.
So far Google has been able to fetch this new file correctly and not show any "couldn't fetch errors" as it did with the others.
(I've set the new sitemap.xml file to 644 permissions.)

[mc12345678]

Thanks for your reply :)
Yes, your are correct, the .htaccess file is included in this plugin and sits inside the sitemap folder that has the .xml files the plugin generates in admin.
As for the sitemap file, I ended up having to generate one from this link www.xml-sitemaps.com and use that instead and removed this plugin's generated .xml files.
So far Google has been able to fetch this new file correctly and not show any "couldn't fetch errors" as it did with the others.
(I've set the new sitemap.xml file to 644 permissions.)

Does your site have linked product? If so, does this "other tool" account for linked product? (basically does it only provide a single url for a product that has multiple urls available for it where they all have the same canonical url?

A program that does not account for linked product on a site that has linked product is not doing you any favors.

As to the indexing "issue(s)" to which you were referring, you need only provide the domain-name/subfolder-if-any/sitemap.xml file. That file is setup to support further reading and finding of information to occur. So not sure what was meant by only reading some of the files when there is only one file to be referenced.

Now, the content within each of those other generated files, sure that will be read/processed when/as necessary as determined by the search engine. Also, considering that those engines work more towards content than anything, if the same content is across multiple items, then the "variety" of the items is reduced and the number of indexed items may decrease over time. I too would not consider myself an SEO expert. I have seen on a rescue related site where each rescued animal has a different description but over time the number of indexed links has been seen to decrease. The point being that the search engine has its reasons and means to index and it may not match with the total number of items in the list that you see...

[barco57]

.htacess files are not supposed to load in the browser by direct access. They control the behavior of requests for assets within the folder. So of course going to www.mysite.com.au/sitemap/.htaccess gets you a 403 error

# deny *everything*
<FilesMatch ".*">
  Order Allow,Deny
  Deny from all
</FilesMatch>

# but now allow just *certain* necessary files:
<FilesMatch ".*\.(xml|xml\.gz)$" >
  Order Allow,Deny
  Allow from all
</FilesMatch>

Those rules only allow .XML or .xml.gz files to be accessed with in that folder, that's it, its only purpose