hack recovery - lessons learnt

[balihr]

One of my clients recently got hacked so I wanted to share the story and lessons learnt, and maybe give a heads-up to everyone here to protect your websites better.

This was a case of a very sophisticated Magecart (digital skimming) attack that successfully bypassed standard server-side malware scanners that are included in this hosting package. The automated scanners did find and quarantined the PHP backdoors the attackers planted, but they completely missed the active data-skimming payloads so credit card details were still being read on the checkout page and sent to the attacker.

Why did the scanners fail?
Steganography. The malicious javascript code was hidden inside a SVG file (favicon.svg) and the main skimming payload was placed in a webp file (logo.webp). Both files pass standard security checks with flying colors unless you really take a close look. And finally, a few lines were added into the templates JS file, allowing the XSS attack to begin.

WHAT YOU CAN DO

#1 Ruthlessly revoke 3rd-party server access

The entry point for this breach wasn't a complex zero-day vulnerability in the code or anything fancy - it was a forgotten, active FTP account belonging to a third-party SEO/marketing agency. We will never know if it was their retaliation for moving to another agency or if their system got compromised and accidentally leaked this information. The attackers used those old credentials and walked right through the front door.

Audit your users: regularly review all FTP, SFTP, SSH, and control panel accounts. Enable 2FA whereever it's available
The principle of least privilege: if a contractor, plugin developer, or agency has finished their scope of work, permanently delete or disable their access immediately.
Use temporary credentials: Never share your root or primary admin credentials. Create isolated accounts with access limited strictly to the directories the third party needs. Set a calendar reminder to kill the access when the job is done.

#2 Run routine manual code comparisons

Automated scanners look for known malicious signatures, meaning they will miss custom obfuscated code or subtle modifications to your core files. You need to verify your production environment against a known good baseline.

Download Production Locally: Periodically pull down your live site files to your local development environment.
Diff the Directories: using an IDE like PHPStorm, or a tool like WinMerge or BeyondCompare, you can run a "Compare Directories" action against a clean, untouched version of your platform (e.g., a fresh Zen Cart release combined with your known good custom template files).
Focus on the frontend: pay special attention to changes in your /includes/templates/, /jscript/, and /images/ directories. If a .js file has unexplained new lines, or an .svg contains <script> tags, you have a breach.

#3 Establish File Integrity Checks (SHA Checks)

A SHA checksum is a cryptographic fingerprint of a file. Even if an attacker adds a single invisible space to a JavaScript file or appends hidden code to a WebP image, the file's SHA fingerprint will completely change. By comparing current fingerprints against a safe baseline, you can instantly spot tampered files.
There are numerous tutorial online how to do these SHA checks so I won't go into details on how to do this.

CONCLUSION

Don't trust the "Clean" scan. Automated malware scanners are a necessary layer of defense, but they are not golden and there are numerous ways of bypassing them. A server can report a 100% clean scan while actively leaking credit card data to a remote server.

Your site is NOT set-and-forget. You need to be active and pro-active, you must manage user permissions and regularly inspect your website - just like you're careful about who has your house keys and how you regularly inspect your car for flaws...

If your site has been hacked, there are many existing threads describing how to recover from hacks. My goal here is simply to raise awareness about how these specific attacks operate and how to lock down your environment proactively in hopes you don't need to get to the recovery part.

[lat9]

Thanks for the detailed posting, @balihr!

[chadlly2003]

I've run into this problem a few times, and it's definitely not fun. I originally thought I had all my i's dotted and t's crossed, but it turns out there were still a few things I needed to address. I added Sucuri to my websites in hopes of preventing it from happening again.

[torvista]

Again, thanks for spending so much time detailing the issue.

I do find I have a different point of view in that I consider the production site a stable backup of my development site!
i.e. one gets fiddled with on a daily basis, the other is making money and gets touched only when all is finalised on the dev site.
The two are compared daily file-by-file to download backups/every time I upload anything, so spotting additions/modifications is instant.

Also the idea of letting a 3rd party loose on a production site is unthinkable. They should be given limited access to a copy, with customer and order details removed.