AbuseIPDB Integration module

[johnjlarge]

I'm not sure if the caching is working correctly. I've woken up just now & almost exhausted my API limit for the day at 7:47am

I just enabled debug mode to check the logs & it would seem it's making a fresh call even for the same ip.

From the bottom of the first debug log

[25-May-2023 07:46:22 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:22 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:22 Europe/London] IP 54.236.1.11 blocked from API call

and then the second

[25-May-2023 07:46:39 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:39 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:39 Europe/London] IP 54.236.1.11 blocked from API call

the third

[25-May-2023 07:46:43 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:44 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:44 Europe/London] IP 54.236.1.11 blocked from API call

So it would seem they are all the same IP address but they are looking up each time if I'm not mistaken.

[johnjlarge]

I'm not sure if the caching is working correctly. I've woken up just now & almost exhausted my API limit for the day at 7:47am

I just enabled debug mode to check the logs & it would seem it's making a fresh call even for the same ip.

From the bottom of the first debug log

[25-May-2023 07:46:22 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:22 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:22 Europe/London] IP 54.236.1.11 blocked from API call

and then the second

[25-May-2023 07:46:39 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:39 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:39 Europe/London] IP 54.236.1.11 blocked from API call

the third

[25-May-2023 07:46:43 Europe/London] Checking cache for IP: 54.236.1.11
[25-May-2023 07:46:44 Europe/London] API call made for IP: 54.236.1.11 with score: 63
[25-May-2023 07:46:44 Europe/London] IP 54.236.1.11 blocked from API call

So it would seem they are all the same IP address, but they are looking up each time if I'm not mistaken.

Just another observation on this, could it be that known spiders are prevented from creating a session, so the IP isn't cached in the session cache? I could be way off, but my initial investigation is that this IP is pinterestbot which I have as a robot in my zen cart who's online/admin robot definitions.

[webchills]

Just another observation on this, could it be that known spiders are prevented from creating a session, so the IP isn't cached in the session cache? I could be way off, but my initial investigation is that this IP is pinterestbot which I have as a robot in my zen cart who's online/admin robot definitions.

Known spiders are definitively prevented from creating sessions

As you are not using the page_not_found redirect, you could change in the observer class from

// Do not execute the check for the 'page_not_found' page
		if ($current_page_base == 'page_not_found') {
			return;
		}

to

// Do not execute the check for known spiders
		if (isset($spider_flag) && $spider_flag === true) {
			return;
		}

to disable the whole thing for known spiders

I'm not using this on a live site at the moment so cannot really test if its working

[johnjlarge]

Known spiders are definitively prevented from creating sessions

As you are not using the page_not_found redirect, you could change in the observer class from

// Do not execute the check for the 'page_not_found' page
		if ($current_page_base == 'page_not_found') {
			return;
		}

to

// Do not execute the check for known spiders
		if (isset($spider_flag) && $spider_flag === true) {
			return;
		}

to disable the whole thing for known spiders

I'm not using this on a live site at the moment so cannot really test if its working

I've changed the code as suggested, but I'll have to wait until tomorrow when my api quota resets to see if it works for spiders. If it works that is quite an elegant fix as I manually block bad bot useragents via .htaccess before they load anything from my site.

[marcopolo]

Known spiders are definitively prevented from creating sessions

As you are not using the page_not_found redirect, you could change in the observer class from

// Do not execute the check for the 'page_not_found' page
		if ($current_page_base == 'page_not_found') {
			return;
		}

to

// Do not execute the check for known spiders
		if (isset($spider_flag) && $spider_flag === true) {
			return;
		}

to disable the whole thing for known spiders

I'm not using this on a live site at the moment so cannot really test if its working

Great solution this modification in the code introduces an additional check to prevent unnecessary IP abuse checks.

If the current page is a 'page_not_found', the IP abuse check will be skipped.
If the visitor is a known web spider or bot, the IP abuse check will also be skipped.

This reduces unnecessary API calls to AbuseIPDB when traffic comes from known web spiders or bots, which are usually harmless.

// Do not execute the check for the 'page_not_found' page or for known spiders
if ($current_page_base == 'page_not_found' || (isset($spider_flag) && $spider_flag === true)) {
    return;
} 

[johnjlarge]

Great solution this modification in the code introduces an additional check to prevent unnecessary IP abuse checks.

If the current page is a 'page_not_found', the IP abuse check will be skipped.
If the visitor is a known web spider or bot, the IP abuse check will also be skipped.

This reduces unnecessary API calls to AbuseIPDB when traffic comes from known web spiders or bots, which are usually harmless.

// Do not execute the check for the 'page_not_found' page or for known spiders
if ($current_page_base == 'page_not_found' || (isset($spider_flag) && $spider_flag === true)) {
    return;
} 

I've changed the modification to this to exclude spiders & 404. Just have to wait until tomorrow to check if block caching is now fully working, but on test mode all seems well. This mod seems to be evolving quickly, but it will do wonders to keep the worst offenders away & reduce my manual blocking workload massively.

Thank you all.

[johnjlarge]

Also, as an aside in case anyone else wants to include an error message on their block page, I'm still serving a 403 but now echoing a browser message as follows.

   header('HTTP/1.0 403 Forbidden');
                echo 'You are forbidden! Your IP Address is marked as malicious in the abuseipdb.com database';
                zen_exit();

[dbltoe]

Any chance we can get a github link to avoid having to add, delete, rinse, repeat?

[marcopolo]

Any chance we can get a github link to avoid having to add, delete, rinse, repeat?

Here is the GitHub link: https://github.com/CcMarc/AbuseIPDB.git