Arbitrary Code Execution via Malicious `pickle` Payload in Hugging Face Models
Overview
A widespread supply chain vulnerability affects the AI/ML ecosystem through the use of the `pickle` serialization format for distributing model weights. Threat actors can craft a malicious model file (e.g., `pytorch_model.bin`) and upload it to public repositories like the Hugging Face Hub. The `pickle` format, used by default in libraries like PyTorch for saving and loading model objects, is not secure and can be instrumented to execute arbitrary code upon deserialization. When a developer or an automated MLOps pipeline downloads and loads the poisoned model using a standard function like `torch.load()`, the embedded malicious code is executed. This provides the attacker with remote code execution capabilities on the machine loading the model, which could be a developer's laptop, a training server, or a production inference endpoint. The impact is severe, allowing for theft of sensitive data, credentials, and proprietary models, or using the compromised infrastructure as a pivot point for further attacks. This vulnerability class highlights the critical need for a secure model serialization format. The development and adoption of the `safetensors` format is a direct response to this threat, providing a safer alternative that only allows for the serialization of tensor data, preventing code execution.
Affected Systems
Testing Guide
1. Create a malicious pickle file that executes a simple command, e.g., creates a file or prints a message. ```python import pickle import os class MaliciousCode: def __reduce__(self): return (os.system, ('touch /tmp/pwned',)) with open('malicious.pkl', 'wb') as f: pickle.dump(MaliciousCode(), f) ``` 2. In a separate, safe environment, attempt to load this file with `pickle.load()` or `torch.load()`. 3. Check if the file `/tmp/pwned` was created. If so, your loading process is vulnerable.
Mitigation Steps
1. **Use `safetensors`:** Prioritize downloading and using models in the `.safetensors` format whenever possible. It is designed to be secure against arbitrary code execution. 2. **Scan Models:** Use tools like `picklescan` or Hugging Face Hub's built-in malware scanner to inspect model files before loading them. 3. **Use Safe Loading in PyTorch:** When loading pickle-formatted models is unavoidable, use `torch.load(file, weights_only=True)` (available in PyTorch 2.1+) to prevent deserialization of arbitrary code. 4. **Vet Model Sources:** Only use models from trusted, verified organizations and creators. Scrutinize models with low download counts or from unknown sources.
Patch Details
Mitigation is available via community best practices (using safetensors) and new safe-loading features in frameworks like PyTorch 2.1+.