Zen Cart Logo
Forums / General Questions / Removing crawlers, spyders, bots from Who's Online?

Removing crawlers, spyders, bots from Who's Online?

Locked

Views: 1,992

Results 1 to 10 of 10
This thread is locked. New replies are disabled.
1 Dec 2008, 6:10 PM
#1
rcrosier avatar

rcrosier

Zen Follower

Join Date:
Dec 2006
Location:
Vermont
Posts:
128
Plugin Contributions:
0

Removing crawlers, spyders, bots from Who's Online?

Hi,

Is it possible to filter OUT all the spyders, googlebots, crawlers, etc., from the "Who's Online" page?

I don't really want or need to see these all the time, and I'd like to filter them out, so I only see the most likely customers online.

is there any relatively easy code change to NOT show all the PINK items?

Thanks.

VTstuff.com, Quality gifts and products from Vermont.

1 Dec 2008, 8:50 PM
#2
stevesh avatar

stevesh

Black Belt

Join Date:
Feb 2005
Location:
Lansing, Michigan USA
Posts:
19,793
Plugin Contributions:
2

Re: Removing crawlers, spyders, bots from Who's Online?

Don't know what 'the PINK items' are, but have you considered just turning off the sidebox? Visitors don't care who's online, and you can see who's visiting from Admin. It's just screen clutter to me.

1 Dec 2008, 9:31 PM
#4
afo avatar

afo

Totally Zenned

Join Date:
Aug 2004
Location:
New York City
Posts:
6,800
Plugin Contributions:
0

Re: Removing crawlers, spyders, bots from Who's Online?

gjh42:

I believe the OP is talking about the admin page.

I think so, too, and I don't know of any way to turn that off.

Mary Ellen
I came; I saw; I Zenned
Taking over the world... one website at a time
Make sure brain is engaged before putting mouth in gear... or fingers to keyboard.

Holzheimer
Fan Odyssey

1 Dec 2008, 11:05 PM
#5
tacoma avatar

tacoma

Zen Follower

Join Date:
Dec 2007
Posts:
125
Plugin Contributions:
0

Re: Removing crawlers, spyders, bots from Who's Online?

I'd like to know how it decides who's a bot and who isn't.

I'd like to know if I can manually add known bot ips to a bot list?

I get a handful of bots that are given session ids because they are treated as a human user, and that bothers me.

For example there are some MSN bots that don't get counted as bots even though the log shows them to be. Host: msnbot-65-55-109-97.search.msn.com.

2 Dec 2008, 2:46 AM
#6
kiddo avatar

kiddo

Totally Zenned

Join Date:
Jul 2006
Location:
SF Bay Area
Posts:
837
Plugin Contributions:
1

Re: Removing crawlers, spyders, bots from Who's Online?

About line 251 in admin/whos_online.php find:

<?php $ip_array = array(); $d=0; while (!$whos_online->EOF) { $time_online = (time() - $whos_online->fields['time_entry']); And add two line like this: <?php $ip_array = array(); $d=0; while (!$whos_online->EOF) { $bot=zen_check_bot($whos_online->fields['session_id']); if($bot){$whos_online->MoveNext();continue;} $time_online = (time() - $whos_online->fields['time_entry']); That will make the pinkies not show.
2 Dec 2008, 1:17 PM
#7
rcrosier avatar

rcrosier

Zen Follower

Join Date:
Dec 2006
Location:
Vermont
Posts:
128
Plugin Contributions:
0

Re: Removing crawlers, spyders, bots from Who's Online?

Thanks to all who responded.

The response from Kiddo did the trick. I appreciate your input greatly.

VTstuff.com, Quality gifts and products from Vermont.

3 Dec 2008, 6:02 AM
#8
ajeh avatar

ajeh

Oba-san

Join Date:
Sep 2003
Location:
Ohio
Posts:
62,757
Plugin Contributions:
1

Re: Removing crawlers, spyders, bots from Who's Online?

Not showing the bots isn't making them not get sessions ... :eek:

Adding bots to the /includes/spiders.txt is ... via their User Agent ...

The problem is the bots that cheat and do not pass this information ... for those that you see, if you can, complain to the folks that send them and tell them to get with the program ... maybe, they will listen ...

NOTE: To see the "pink" bots ... you do need to store your sessions to the database ...

Linda McGrath
If you have to think ... you haven't been zenned ...

**Did YOU buy the Zen Cart Team a cup of coffee and a donut today? Just click here to support the Zen Cart Team!!**

Are you using the latest? Perhaps you've a problem that's fixed in the latest version: [Upgrade today!]
Officially PayPal-Certified! Just click here

Try our Zen Cart Recommended Services - Hosting, Payment and more ...
Signup for our Announcements Forums to stay up to date on important changes and updates!

4 Dec 2008, 12:51 PM
#9
rcrosier avatar

rcrosier

Zen Follower

Join Date:
Dec 2006
Location:
Vermont
Posts:
128
Plugin Contributions:
0

Re: Removing crawlers, spyders, bots from Who's Online?

All I wanted was to not "see" them in my "who's online" listing...
But is it causing me some hidden problem(s) by doing this?

I have not noticed anything bad since I implemented this code change to filter them out...

When you add them to your spiders.txt file, do you use the entire domain name? If not, what part do you add?

Thanks.

VTstuff.com, Quality gifts and products from Vermont.

4 Dec 2008, 3:07 PM
#10
ajeh avatar

ajeh

Oba-san

Join Date:
Sep 2003
Location:
Ohio
Posts:
62,757
Plugin Contributions:
1

Re: Removing crawlers, spyders, bots from Who's Online?

View the spiders.txt file and try looking for:
msnbot
yahoo
google

to give you an idea how the User Agent is utilized for this ...

The whos_online will show the User Agent that identifies these ... when that code blocking spiders is not active ... :smile:

You are not hurting anything by not "seeing" these ...

Note: if you click to sort the whos_online by the Name, it will group Guests, Customers and Spiders for you ...

Linda McGrath
If you have to think ... you haven't been zenned ...

**Did YOU buy the Zen Cart Team a cup of coffee and a donut today? Just click here to support the Zen Cart Team!!**

Are you using the latest? Perhaps you've a problem that's fixed in the latest version: [Upgrade today!]
Officially PayPal-Certified! Just click here

Try our Zen Cart Recommended Services - Hosting, Payment and more ...
Signup for our Announcements Forums to stay up to date on important changes and updates!