Malicious AI Model on Hugging Face Hub Leads to Supply Chain Compromise via Pickle Deserialization
Overview
A widespread supply chain attack was identified where threat actors uploaded trojanized machine learning models to the Hugging Face Hub. The attack vector leverages the common practice of using Python's `pickle` format for serializing and distributing PyTorch (`.pth`, `.pt`) and other framework models. The `pickle` module is notoriously insecure as deserializing a specially crafted pickle file can lead to arbitrary code execution. In this incident, attackers forked popular, legitimate models and inserted a malicious payload into one of the model's layers, saving it in the `pickle` format. The payload was typically a small Python class with a `__reduce__` method that, upon deserialization via `torch.load()`, would execute code to download and run a second-stage malware implant. This implant was designed to steal sensitive information from developer environments, including AWS credentials, SSH keys, and cryptocurrency wallet files. Because models are often downloaded and executed automatically as part of MLOps pipelines or research workflows, the attack had a high success rate. The incident exposed critical gaps in the validation and scanning of user-uploaded content on public model repositories and the dangerous default of trusting model weights from unverified sources.
Affected Systems
Testing Guide
1. Identify all `torch.load()` or `pickle.load()` calls in your codebase that load model files from external sources. 2. Install a model scanner: `pip install picklescan`. 3. Run the scanner on your local model cache directory (e.g., `~/.cache/huggingface/hub`): `picklescan -p /path/to/models`. 4. The scanner will report any files that trigger dangerous `__reduce__` calls during deserialization. Any findings should be considered indicative of a potential compromise.
Mitigation Steps
1. **Use SafeTensors:** Whenever possible, use the `safetensors` format for loading models. It is a secure alternative to `pickle` that does not allow for arbitrary code execution. 2. **Scan Models:** Use tools like `picklescan` or other vulnerability scanners to check model files for malicious payloads before loading them. 3. **Verify Model Sources:** Only download models from trusted and verified creators on Hugging Face or other repositories. Be wary of recently uploaded models or forks with low download counts. 4. **Isolate Model Loading:** Load and process untrusted models in a sandboxed, network-isolated environment to limit the potential impact of a compromise.
Patch Details
This is an attack pattern, not a specific software vulnerability. Mitigation relies on user awareness and tooling. Hugging Face has improved scanning but cannot guarantee safety for all formats.