Malicious Model Backdoor on Hugging Face Hub via Unsafe `pickle` Deserialization
Overview
A significant supply chain attack vector was demonstrated where threat actors upload seemingly legitimate pre-trained models to the Hugging Face Hub that contain malicious backdoors. The vulnerability does not lie within the Hugging Face platform itself, but rather in the standard practice of using Python's `pickle` module for model serialization, which is inherently insecure. Many models, particularly older ones or those from less-established sources, are distributed as `pytorch_model.bin` files, which are often pickled Python objects. An attacker can craft a model file containing a malicious `__reduce__` method. When a victim developer downloads and loads this model using standard library calls like `torch.load()` or `transformers.AutoModel.from_pretrained()`, the pickled payload is deserialized, leading to arbitrary code execution on the developer's machine. This attack has been used to exfiltrate environment variables (including AWS keys, API tokens), establish reverse shells, and deploy ransomware. The incident underscores the critical need for developers to treat pre-trained model weights as untrusted executable code and to adopt safer serialization formats.
Affected Systems
Testing Guide
1. **Check File Format:** When browsing the Hugging Face Hub, check the 'Files and versions' tab for a model. If it only contains `pytorch_model.bin` and lacks a `model.safetensors` file, it carries a higher risk. 2. **Local Scan:** Before loading a `.bin` or `.pkl` file, run a scanner on it: `picklescan --path /path/to/your/model/`. 3. **Observe Suspicious Imports:** The scanner will flag dangerous imports like `os`, `subprocess`, or `socket` within the pickle stream, which are strong indicators of a malicious model.
Mitigation Steps
1. **Use SafeTensors:** Prioritize loading models exclusively in the `safetensors` format. This format is designed for security and performance, and it does not allow for arbitrary code execution. Always specify `use_safetensors=True` when possible. 2. **Scan Models:** Before loading any model, especially from an untrusted source, use security scanning tools like `picklescan` to inspect the files for malicious payloads. 3. **Verify Provenance:** Only download models from trusted and verified organizations on Hugging Face. Check the model card for details, download counts, and community feedback. 4. **Sandbox Execution:** Load and run new models for the first time in a sandboxed, network-isolated environment to contain any potential malicious activity.
Patch Details
This is an ecosystem-wide issue, not a specific software vulnerability. Mitigation relies on user awareness and adopting safer formats like safetensors.