AbuseIPDB Integration module

[johnjlarge]

Just a thought.

Could it be that the spiders detection/do not allow sessions for spiders code happens further down in the code stack, so the abuseipdb plugin does its api lookup/redirect before the useragent is detected as a spider from the spiders.txt file?

[marcopolo]

Just a thought.

Could it be that the spiders detection/do not allow sessions for spiders code happens further down in the code stack, so the abuseipdb plugin does its api lookup/redirect before the useragent is detected as a spider from the spiders.txt file?

I checked that I think it's good from the program flow I referenced:
https://docs.zen-cart.com/dev/code/program_flow/

Set up and start session if valid session is above where this module comes in which is: NOTIFY_HTML_HEAD_START (the /includes/templates/common/html_header.php) unless I'm missing something. I tried going further down the list which did not resolve the issue.

[marcopolo]

Recap and Solution I Implemented in v2.0.7 for the Meantime:

Problem:
During testing, I faced difficulties with the spider detection feature in Zen Cart. Specifically, the utilization of the $spider_flag variable did not produce the expected results, particularly in generating logs after detecting a spider using:

if (isset($spider_flag) && $spider_flag === true) 

Solution Implemented:
To resolve this issue, I decided to clone the spider detection code from Zen Cart and integrate it directly into the module functions file. This allowed me to incorporate the spider detection functionality without relying on Zen Cart's $spider_flag variable. As a result, the spider detection now works as intended within my custom code.

function checkSpiderFlag() {
    if (defined('SESSION_BLOCK_SPIDERS')) {
        $user_agent_abuseipdb = '';
        if (isset($_SERVER['HTTP_USER_AGENT'])) {
            $user_agent_abuseipdb = strtolower($_SERVER['HTTP_USER_AGENT']);
        }
        $spider_flag_abuseipdb = false;
        if (!empty($user_agent_abuseipdb)) {
            $spiders_abuseipdb = file(DIR_WS_INCLUDES . 'spiders.txt');
            for ($i = 0, $n = sizeof($spiders_abuseipdb); $i < $n; $i++) {
                if (!empty($spiders_abuseipdb[$i]) && substr($spiders_abuseipdb[$i], 0, 4) != '$Id:') {
                    if (is_integer(strpos($user_agent_abuseipdb, trim($spiders_abuseipdb[$i])))) {
                        $spider_flag_abuseipdb = true;
                        break;
                    }
                }
            }
        }
        return $spider_flag_abuseipdb;
    }
    return false; 

Although the current solution successfully resolves the logging problem, I am interested in understanding why using the original $spider_flag variable did not produce the expected outcome. If anyone has any insights, suggestions, or guidance that could be a potential resolution that would allow me to revert back to using the $spider_flag in the future, please do let me know!

Thank you for your time and assistance.

[johnjlarge]

Recap and Solution I Implemented in v2.0.7 for the Meantime:

Problem:
During testing, I faced difficulties with the spider detection feature in Zen Cart. Specifically, the utilization of the $spider_flag variable did not produce the expected results, particularly in generating logs after detecting a spider using:

if (isset($spider_flag) && $spider_flag === true) 

Solution Implemented:
To resolve this issue, I decided to clone the spider detection code from Zen Cart and integrate it directly into the module functions file. This allowed me to incorporate the spider detection functionality without relying on Zen Cart's $spider_flag variable. As a result, the spider detection now works as intended within my custom code.

function checkSpiderFlag() {
    if (defined('SESSION_BLOCK_SPIDERS')) {
        $user_agent_abuseipdb = '';
        if (isset($_SERVER['HTTP_USER_AGENT'])) {
            $user_agent_abuseipdb = strtolower($_SERVER['HTTP_USER_AGENT']);
        }
        $spider_flag_abuseipdb = false;
        if (!empty($user_agent_abuseipdb)) {
            $spiders_abuseipdb = file(DIR_WS_INCLUDES . 'spiders.txt');
            for ($i = 0, $n = sizeof($spiders_abuseipdb); $i < $n; $i++) {
                if (!empty($spiders_abuseipdb[$i]) && substr($spiders_abuseipdb[$i], 0, 4) != '$Id:') {
                    if (is_integer(strpos($user_agent_abuseipdb, trim($spiders_abuseipdb[$i])))) {
                        $spider_flag_abuseipdb = true;
                        break;
                    }
                }
            }
        }
        return $spider_flag_abuseipdb;
    }
    return false; 

Although the current solution successfully resolves the logging problem, I am interested in understanding why using the original $spider_flag variable did not produce the expected outcome. If anyone has any insights, suggestions, or guidance that could be a potential resolution that would allow me to revert back to using the $spider_flag in the future, please do let me know!

Thank you for your time and assistance.

Testing here, and it's working really well with the new spiders detection code. Saving a lot of API calls on my site. Also, I've set my threshold at 30% as most of my customers score a 0 threat score, so I'm currently blocking about 10% of hits to the server. I've gone through the logs manually checking bad IPs and there are some real nasty ones in there. This plugin will save a lot of server resources and turn away some of the worst bad bots & crawlers. Invaluable to small shop owners.

Thank you for all the hard work :)

[carlwhat]

Testing here, and it's working really well with the new spiders detection code. Saving a lot of API calls on my site. Also, I've set my threshold at 30% as most of my customers score a 0 threat score, so I'm currently blocking about 10% of hits to the server. I've gone through the logs manually checking bad IPs and there are some real nasty ones in there. This plugin will save a lot of server resources and turn away some of the worst bad bots & crawlers. Invaluable to small shop owners.

Thank you for all the hard work :)

having recently implemented this plugin on a couple of sites, i can truly see its benefit.

it is a GREAT addition to the zen-cart plugin repository.

thanks to @marcopolo for this contribution!