Remote Code Execution via Maliciously Crafted PyTorch Models on Hugging Face Hub
Overview
A significant supply chain vulnerability affects AI developers using models from public repositories like the Hugging Face Hub. The vulnerability resides not in the AI frameworks themselves, but in the serialization format used for model weights. Many older and some current models use Python's `pickle` format (`pytorch_model.bin`), which is known to be unsafe for deserializing untrusted data. The `pickle` module can execute arbitrary code embedded within the data stream during deserialization. Attackers can craft a malicious PyTorch model, replace legitimate layers or tensors with a `pickle` payload, and upload it to the Hub disguised as a popular or useful model. When a developer downloads this model and loads it using `torch.load()`, the payload executes on their machine. This can lead to a complete system compromise, including data theft, ransomware deployment, or using the victim's machine for crypto mining. Researchers demonstrated this by creating a proof-of-concept model that, upon loading, established a reverse shell back to an attacker's server. This attack highlights the critical need for secure model serialization formats and developer awareness when sourcing pre-trained models from the internet. The `safetensors` format was developed as a secure alternative, as it does not have an associated code execution risk.
Affected Systems
Testing Guide
1. **DO NOT USE on a production or sensitive machine.** In a secure, isolated VM, download a known malicious PoC model (if available) or craft one. 2. Write a simple Python script to load the model using `torch.load('path/to/malicious_pytorch_model.bin')`. 3. Monitor network traffic for unexpected outbound connections (e.g., a reverse shell) or check the filesystem for unexpected file creation. If such activity occurs, the loading process is vulnerable.
Mitigation Steps
1. **Prefer `safetensors`**: Prioritize downloading and using models that are available in the `.safetensors` format. This is the most effective mitigation. 2. **Scan Models Before Loading**: Use tools like `picklescan` to scan model files for malicious `pickle` payloads before loading them into memory. 3. **Isolate Model Loading**: Load new or untrusted models in a sandboxed, isolated environment with no network access or sensitive data to limit the potential impact of a compromise. 4. **Verify Model Source**: Only use models from trusted and verified creators on platforms like Hugging Face. Check for signs of legitimacy like download counts, likes, and community discussion.
Patch Details
This is a fundamental risk with the pickle format. The mitigation is to use safer formats like safetensors, which is supported by PyTorch and Hugging Face.