Credential Theft via Malicious PyPI Packages Targeting AI/ML Developers
Overview
A sophisticated supply chain attack was identified involving over 30 malicious packages uploaded to the Python Package Index (PyPI). These packages used typosquatting to mimic popular AI/ML libraries, such as `torch-utils`, `transformers-core`, `tensorflow-addons`, and `huggingface-cli`. The malicious packages contained code within their `setup.py` files that executed upon installation via `pip install`. This setup script was designed to scan the developer's machine and CI/CD environment for sensitive information. It specifically searched for environment variables containing AWS access keys (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), SSH private keys in the `~/.ssh` directory, and Kubernetes configuration files (`~/.kube/config`). Upon finding these credentials, the script would exfiltrate them via a base64-encoded payload to a remote command-and-control (C2) server using DNS tunneling to evade network firewalls. The attack campaign, dubbed "ML-Leach" by researchers at JFrog Security, highlights the significant risk to organizations developing AI applications, as a single compromised developer machine or CI runner could lead to a full breach of cloud infrastructure and model repositories.
Affected Systems
Testing Guide
1. Review recent additions to your project's dependencies for packages with names similar to popular libraries but with slight misspellings or extra suffixes. 2. Check your shell history (`history | grep "pip install"`) and CI/CD logs for installations of suspicious packages. 3. Monitor outbound network traffic from development machines and build servers for unusual DNS requests or connections to unknown endpoints, especially during package installation phases.
Mitigation Steps
1. **Audit Dependencies:** Use tools like `pip-audit` or commercial solutions to scan `requirements.txt` and `pyproject.toml` files for known malicious or vulnerable packages. 2. **Use a Private Registry:** Mirror trusted packages to a private, internal package registry (e.g., Artifactory, Nexus) and configure `pip` to only use that registry. 3. **Lock Dependencies:** Use lock files (`poetry.lock`, `pipfile.lock`, or `requirements.txt` with hashed versions) to ensure deterministic builds and prevent unexpected package updates. 4. **Educate Developers:** Train developers on the dangers of typosquatting and to be vigilant when installing new packages, verifying package names against official sources.
Patch Details
The malicious packages have been removed from PyPI by the security team, but new variations may appear. Mitigation relies on user vigilance and security best practices.