Bots and Spiders

**johns4y** · 16 Oct 2022, 11:51 PM

jaysmodelkits.com
ZC v.1.5.7d soon to be 1.5.8 (wasn't expecting another major update so soon, looks like I'm going to have to remortgage the house, again :op

I've noticed an alarming uptick in activity from bots and spiders crawling my website. Now this should be a good thing, correct? But I am a little suspicious as to what pages they are looking at. I have seen bots from MSN looking at customers order history as soon as a new order comes through. I would be interested to know why they should be looking at these pages and what exactly they are able to see. I assume the details of the order can only be seen by the customer?

On the subject of bots and spiders, I've noticed that, even though I have 'Exclude Spiders' option as Yes, I still see a lot of them coming up with multiple pages assigned to the same IP address. Again, just interested to know why some Spiders/Bots are displayed and others aren't. If I have the option to exclude set to no additional spiders appear in pink.
Cheers, John.

**dbltoe** · 17 Oct 2022, 12:56 AM

Knowing your current mods and settings, I would not think about update until you get some of your payment modules working with 1.5.7d. See your display logs.

NEVER complain that your site can be found on the Internet. That's the ultimate goal. Having to deal with a few script kiddies trying to get places they can't is just part of the "reward" for exposure. Zen Cart didn't get PCI certified by letting folks get in to someone's order history.

Remember that it is easy to spoof anything and view the whos on line as to who is putting items in the cart and just what items they are looking at. Then, adjust your advertising/social media accordingly.

**johns4y** · 17 Oct 2022, 03:33 AM

Originally Posted by dbltoe

NEVER complain that your site can be found on the Internet.

Who's complainin'? I was just interested to know what the Bots were up to and why. Regarding the other matter of payment modules, I have two that I know of that are in use, PayPal and PayPal Express Checkout. As far as I know they are working just fine. The option for New Zealand customers to pay via bank transfer has a tendency to throw up the occasional fault log, I would have no idea how to put it right. It's a legitimate method of payment and one I don't intend to remove.

**johns4y** · 17 Oct 2022, 10:10 AM

Name: Screen Shot 2022-10-17 at 22.05.13.jpg
Views: 145
Size: 42.1 KB

If you can the see screenshot above, this is the sort of thing I'm talking about. First the MSN Bot visits the login page, then it visits the account history info page for the most recent order.
This is happening a lot now. Is it just a coincidence that the bot is visiting the login page then checking out a recent order? It's almost as if they have the customers info and are looking to see what they have ordered.
Looks a bit dodgy to me.

**CunningStunt2** · 27 Oct 2022, 10:20 AM

Web crawlers, spiders and bots, etc should appear in the who's online page with a pink background. That seems to be a human user which just happens to have msnbot as part of their host name.

The process is also happening in the reverse order of what you posted, the user is trying to access an order history page first, but because the user is not logged in they are immediately redirected to the login page. The screen shows time since last click being one second longer for the order history page so that was the first page this ip tried to visit.