Malicious Model on Hugging Face Hub Leverages Unsafe Pickle Deserialization for Remote Code Execution
Overview
A significant supply chain attack vector was identified affecting developers who download and use pre-trained models from public repositories like the Hugging Face Hub. Attackers were found to upload seemingly benign models, often forks of popular models with minor modifications. However, the model's weight file (`pytorch_model.bin` or similar) was a maliciously crafted pickle file. PyTorch's `torch.load()`, a standard function for loading models, uses Python's `pickle` module internally, which is known to be unsafe for deserializing untrusted data as it can execute arbitrary code. The malicious pickle file contains a payload that is executed on the developer's machine as soon as `torch.load()` is called. This provides the attacker with a persistent foothold, allowing for theft of sensitive data such as API keys, SSH keys, and proprietary code, or using the compromised machine as a pivot point for further attacks. The 'Silent Siren' campaign was a notable instance where researchers at Wiz discovered multiple malicious models that had been downloaded thousands of times before being identified. This vulnerability highlights the critical need for verifying the provenance of AI models and avoiding unsafe serialization formats.
Affected Systems
Testing Guide
1. **Install Scanner**: Install a pickle scanning tool: `pip install picklescan`. 2. **Scan Model Files**: Run the scanner on your local model cache directory (e.g., `~/.cache/huggingface/hub`). Command: `picklescan -p /path/to/your/models`. 3. **Review Findings**: The tool will report any files that contain potentially dangerous opcodes. Any finding of `REDUCE`, which is used to call arbitrary functions, is a strong indicator of a malicious file.
Mitigation Steps
1. **Use SafeTensors**: Always prefer loading models in the `.safetensors` format, which is a secure alternative to pickle. Use `from_safetensors=True` where available. 2. **Scan Models**: Before loading any model, use tools like Hugging Face's built-in malware scanner or third-party tools like `picklescan` to check for malicious payloads. 3. **Vet Model Sources**: Only use models from trusted, verified creators on platforms like Hugging Face. Be wary of recently uploaded models or those from unknown authors. 4. **Sandbox Loading**: If you must load a pickle file from an untrusted source, do so in a heavily sandboxed, isolated environment with no network access or access to sensitive files.
Patch Details
This is a risk inherent to the pickle format, not a bug in PyTorch itself. The solution is to move to safer formats and practices.