Arbitrary Code Execution via Malicious Pickle-Serialized Models on Hugging Face Hub
Overview
A persistent supply chain vulnerability affects the AI ecosystem through the distribution of malicious models on platforms like the Hugging Face Hub. Many models, especially older ones or those from the PyTorch ecosystem, are saved using Python's `pickle` format (`.pkl` or `.bin` files). The `pickle` module is notoriously insecure, as it can execute arbitrary code during the deserialization process. An attacker can craft a model file containing a malicious pickle payload and upload it to the Hub, disguising it as a legitimate fine-tuned model. When an unsuspecting developer or MLOps pipeline downloads and loads this model using `torch.load()` or a similar pickle-dependent function, the embedded payload executes on their machine. This can lead to the theft of credentials (API keys, environment variables), installation of ransomware, or the co-opting of the machine into a botnet for crypto mining or DDoS attacks. While Hugging Face has implemented security scans and heavily promotes the safer `safetensors` format, the continued existence and use of pickle-based models remain a significant risk. The attack requires no user interaction beyond the standard model-loading process, making it a potent supply chain threat.
Affected Systems
Testing Guide
1. **DO NOT USE A REAL MALICIOUS PAYLOAD**. In a sandboxed environment, create a safe proof-of-concept pickle file that performs a harmless action, like creating a file `touch /tmp/vulnerable`. 2. Use a tool like `pker` to embed this safe payload into a dummy model file. 3. Load the model file using `torch.load()` without any security arguments. 4. Check if the file `/tmp/vulnerable` was created. If it was, your loading process is vulnerable.
Mitigation Steps
1. **Use `safetensors`**: Exclusively use the `safetensors` format for saving, sharing, and loading models. It is a secure alternative that does not allow for arbitrary code execution. 2. **Scan Models**: Use Hugging Face's built-in security scanner or third-party tools like `picklescan` to check for malicious payloads in model files before loading them. 3. **Load from Trusted Sources**: Only download models from verified creators or organizations on the Hugging Face Hub. 4. **Use `weights_only=True`**: When forced to use `torch.load`, use the `weights_only=True` argument (available in newer PyTorch versions) to restrict unpickling to tensors and simple data types, preventing code execution.
Patch Details
Mitigation relies on user adoption of safetensors and safe loading practices. Hugging Face implemented server-side scanning and UI warnings.