bots needing to be added to Spiders.txt

ltx71

If you know of others please add them here...

Datanyze

Can you submit a PR?

Don't understand PR?

Need to add: Ruby

The bot "dispatch" needs to be added

If the user agent is nothing how can we cause those sessions to be considered spider bots?

Session ID: e4ba3f59af80f98706d6051
Host: 115.146.122.
User Agent:

swguy:

Can you submit a PR?

shrimp-gumbo-mmmhhh:

Don't understand PR?

A PR is a Pull Request. This is an action taken using a tool that works with Git. The Zen Cart software is made available through github (https://github.com/zencart/zencart) where it is possible to offer up code changes through making a pull request. Basically you create a branch off of the code to be changes, change the desired code, explain what the change does why it was done etc, then request the Core Zen Cart team to incorporate the code. They (and others) will review and maybe give comment, criticism, advice, and then either more changes get made or the change is accepted as is. And tada, the public have spoken about this open source software.

There are a lot of tools available out there.

Okay, so swguy is asking me to check the file out and put the extra bot names in and then check in?

Join the fun!

So on Spiders.txt I click "Fork this project and edit the file"? or What do i do to check it out to edit it?

Okay, joined GitHub, Downloaded the GitHub desktop, Clicked "Open the File In GitHub Desktop" and it appears to have downloaded the zencart 1.5.7 repository to my computer..... does this get updated when a new commit is made or what do I have to do to always be looking at a fresh 1.5.7? How Do I create a pull request? How do I check out spiders.txt and check it back in? LOL! Maybe I will figure something out.

Moving along.... I edited the spiders.txt in my repository.... moving the bots not in alpha order to alpha order as my first try.... I try to push to origin but I am getting this message "Authentication Failed. You may not have permission to access the repository or the repository may have been archived. Open options and and verify that you're signed in with an account that has permission to access this repository." Okay- how to get permission to work with zencart?

BTW I am user Shrimp-Gumbo

When I am in GitHub and at the zencart project.... I click "New Pull Request" after I do this it pulls up a "Compare changes" screen. I have no idea how to create one after pushing the "New Pull Request".

LOL! Figured it out finally.

It appears that google and other bots are not identifying themselves now??? I am seeing a lot of hits from like: Host: corpnat-104-132-217-95.corp.google.com Where you can tell it is Google from the Host but they do not have a robot name.... I know this is the case because of so little traffic I have.... I believe other "robots" may be hiding too..

Any thoughts?

I am seeing some "bots" in hiding doing specialized searches on my site for brand names.... just searching for brand names....

shrimp-gumbo-mmmhhh:

It appears that google and other bots are not identifying themselves now??? I am seeing a lot of hits from like: Host: corpnat-104-132-217-95.corp.google.com Where you can tell it is Google from the Host but they do not have a robot name.... I know this is the case because of so little traffic I have.... I believe other "robots" may be hiding too..

Any thoughts?

shrimp-gumbo-mmmhhh:

I am seeing some "bots" in hiding doing specialized searches on my site for brand names.... just searching for brand names....
Yup. There's always lots of them out there hiding, masquerading, collecting information surreptitiously.

List to add for next update of Spiders.txt

headlesschrome
alibaba
aliapp
frd-al00
liebaofast
mb2345browser
oppo a33