Forums / Search Engines / spider71.yandex.ru

spider71.yandex.ru

Locked

Views: 1,360

Results 1 to 20 of 35
This thread is locked. New replies are disabled.
10 Dec 2009, 9:53 PM
#1
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

spider71.yandex.ru

The above has been hitting my shop for days and days. It keeps ading to the cart, and pages it seems to target the most are unsubscribe, log in, tell a friend (switched off anyway), as if searching for a vulnerability. I have blocked it in every way I can in the htaccess file, to no avail.

I must admit, despite my research showing it to be a 'search engine', that inspires me with no confidence at all and I think it's dodgy.

Wits end! Can anyone help?

Lorrie
version Zen Cart v1.3.8a/v1.3.8

10 Dec 2009, 9:59 PM
#2
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

See the "Similar Threads" section below for related discussion.

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

10 Dec 2009, 10:00 PM
#3
dfmc avatar

dfmc

New Zenner

Join Date:
Mar 2008
Posts:
29
Plugin Contributions:
0

Re: spider71.yandex.ru

Having the same issue on my end and since I don't market to Russia, let alone anywhere outside my local area (meaning I only market or target a couple smalls towns where I live), seems pointless to have it eating up my bandwidth for something I don't want period.

I couldn't make heads nor tails out of the other threads on this subject, so could someone put it a little simpler terms and instructions for us to stop this yandex thing completely please.

Thanks in advance.

10 Dec 2009, 11:07 PM
#4
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

Non of those 'similar threads' (which I had already thoroughly read through prior to posting having done a search and really 'enjoyed' the little spat midway through the most comprehensive) have been of any help. All these precautions I had already taken.

Never mind then....

10 Dec 2009, 11:11 PM
#5
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

Illusio:

All these precautions I had already taken.Well, if you've already blocked the IP address in your hosting account as explained, then the yandex visits should have already stopped.

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

10 Dec 2009, 11:20 PM
#6
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

I have...and they haven't.

But thanks anyway.

10 Dec 2009, 11:23 PM
#7
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

Then you've either NOT blocked the IP address that's visiting you, or you've done it in a way that's not working for your hosting account.
Have you contacted your hosting company for help in blocking it?

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

10 Dec 2009, 11:24 PM
#8
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

LOL!! Ohhhh yes. In fact in desperation, they came here!

10 Dec 2009, 11:26 PM
#9
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

It's better than it was. It is now 'glorious pink' as quoted by someone else, and it's activity is now more limited. But, it would appear to have tkaen up residence. I think I may just have to live with it for the mo.

10 Dec 2009, 11:55 PM
#10
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

What are the IP addresses listed "in glorious pink"?
What method did you use to block the IP addresses?

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

11 Dec 2009, 2:01 AM
#11
rodg avatar

rodg

Deceased

Join Date:
Jan 2007
Location:
Australia
Posts:
6,263
Plugin Contributions:
4

Re: spider71.yandex.ru

DrByte:

Well, if you've already blocked the IP address in your hosting account as explained, then the yandex visits should have already stopped.

FYI, the Yandex spider, as per previous threads, uses half a dozen or more different IP address on different networks It is very difficult to block by IP address... block one, the come in on another. It isn't even possible to block them by subnet due to the disparate IP's they use.

They ignore the robots.txt file(s), they hit heavy and hard.

Really not a very friendy bot at all.

Cheers
Rod

NOTE: We are sorry that Rod is no longer with us.
We are grateful for all his contributions to the Zen Cart community.

Ozpost - The Ultimate Shipping module for Australian Merchants. Click these links for its Homepage, or Download from Zen-Cart.com.

11 Dec 2009, 3:23 AM
#12
tino_schlegel avatar

tino_schlegel

Totally Zenned

Join Date:
Nov 2007
Location:
Melbourne, Australia
Posts:
532
Plugin Contributions:
0

Re: spider71.yandex.ru

Add the name in the /includes/spiders.txt file. this should prevent the spider from creating sessions add adding stuff to the cart.

00:22:39 0 ¥Guest 77.88.27.25 12:04:26 12:18:53
Time Since Clicked:
00:08:12 ago Session ID: [xxx]
Host: spider27.yandex.ru
User Agent: Yandex/1.01.001 (compatible; Win16; I)

Add "yandex" on a new line at the end of the file.

11 Dec 2009, 3:54 AM
#13
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

Tino, as discussed elsewhere, merely adding 'yandex' to the robots.txt file does NOT stop them from indexing the site. It merely stops them from adding-to-cart.

These folks are complaining that they don't want yandex to even index their site.

So ... assuming yandex doesn't change their User Agent, and assuming THEY never see THIS post, you can add the following to your site if you want yandex to never see anything on your site. USE AT OWN RISK: Implementing this wrong could cause your site to malfunction or not be indexed at all.

Create a new file: /includes/extra_configures/block_yandex_spider.php ... containing ONLY the following code:```

<?php if (stristr(strtolower($_SERVER['HTTP_USER_AGENT']), 'yandex')) { @header("HTTP/1.1 410 Gone"); exit(); } ``` This will cause the yandex spider to see a 410 response, which normal spiders will use to delete the page/site from their database, and then simply see a blank page even if they ignore the 410.

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

11 Dec 2009, 9:16 AM
#14
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

I was not able to 'block' as such, but only able to make them be recognised as spider rather than a customer. So, it is stopping them (I think) from adding to cart etc.

This is the specific IP hitting my site: 95.108.128.241
I added this code to the spider.txt: Yandex/1.01.001

I have seen the post about the php file, I will do it...carefully.

Thanks everyone.

11 Dec 2009, 9:26 AM
#15
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

Done.

Will let you know how it goes...and thank you.

Lorrie :)

11 Dec 2009, 11:35 AM
#16
illusio avatar

illusio

New Zenner

Join Date:
Aug 2008
Location:
West Sussex, UK
Posts:
17
Plugin Contributions:
0

Re: spider71.yandex.ru

Just to let you know, installing the php file was easy, and it works. I have been listing new products and then Twittering, so a lot of activity on my site, including google bots. Since installing the file, have had no trouble with yandex.

Cheers..

11 Dec 2009, 4:46 PM
#17
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

Regarding spiders.txt ...> Illusio:

I added this code to the spider.txt: Yandex/1.01.001
Actually, it's important to do something shorter than that, and do it all lowercase.

Thus, simply adding "yandex" would be the best approach for spiders.txt, and not "Yandex/1.01.001".

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

11 Dec 2009, 4:46 PM
#18
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,515
Plugin Contributions:
176

Re: spider71.yandex.ru

Illusio:

Just to let you know, installing the php file was easy, and it works. I have been listing new products and then Twittering, so a lot of activity on my site, including google bots. Since installing the file, have had no trouble with yandex.

Cheers..

Thanks for confirming that the script is working as desired.

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

11 Dec 2009, 5:38 PM
#19
dfmc avatar

dfmc

New Zenner

Join Date:
Mar 2008
Posts:
29
Plugin Contributions:
0

Re: spider71.yandex.ru

Thanks DrByte!

I followed both steps i.e. yandex to the robots.txt and also created the php file and added it as well. Both seem to be doing their jobs perfectly and they are no longer showing up in my who's online list so far.

The yandex that was on my site was using spider45.yandex.ru just in case anyone else is or was getting that one as well.

Thanks again for all of your hard work and ofcourse the help!

15 Dec 2009, 5:54 PM
#20
gonewild avatar

gonewild

Zen Follower

Join Date:
Apr 2005
Posts:
145
Plugin Contributions:
0

Re: spider71.yandex.ru

Maybe Yandex is not good to have around for any reason?

I've first noticed the spiders presence a couple weeks ago and lately it has been dominating.
Also over the last week sales and adds to cart dropped way off, like to zero for 3 days.
Also I noticed that other bots seemed to not be around.
I added the PHP file as shown in post 13 of this thread last night.
This morning I have no yandex and I have orders coming in.

Would there be a part of the yandex spider that blocks other users?