Malicious Model on Hugging Face Hub Leverages `pickle` Deserialization for Arbitrary Code Execution
Overview
A supply chain attack vector was demonstrated where a malicious actor uploaded a seemingly benign pre-trained model to the Hugging Face Hub, which, upon being loaded, executes arbitrary code on the user's machine. The attack exploits the inherent insecurity of Python's `pickle` serialization format, which is a common method for saving and loading PyTorch models (`.pt`, `.pth` files) and other ML artifacts. The `pickle` format is not secure against erroneous or maliciously constructed data. A malicious actor can craft a model file containing a serialized Python object whose `__reduce__` method is overridden to call arbitrary system commands (e.g., using `os.system` or `subprocess.run`). When a developer or MLOps pipeline downloads this model and loads it using a function like `torch.load()` or `pickle.load()`, the malicious code is executed instantly. This provides the attacker with a persistent backdoor, credentials theft from environment variables, or the ability to poison the model training environment. The incident highlights a critical supply chain risk in the MLOps lifecycle, where trust is often implicitly placed in community-contributed models. The Hugging Face Hub has since implemented enhanced scanning and warnings for models using the pickle format, urging users to switch to safer formats like `safetensors`.
Affected Systems
Testing Guide
1. **Review Codebase:** Search your code for instances of `pickle.load()`, `torch.load()`, `joblib.load()`, and similar functions that load serialized model files. 2. **Check File Extensions:** Identify the types of model files being loaded (e.g., `.pkl`, `.pt`, `.pth`, `.bin`). Files other than `.safetensors` are potentially risky. 3. **Use a Scanning Tool:** Run a tool like `picklescan` against your local model cache or downloaded model files. Command: `picklescan -p /path/to/your/models`. 4. **Observe Scan Output:** The tool will flag any files containing potentially dangerous opcodes. Any finding should be considered a high-risk vulnerability.
Mitigation Steps
1. **Use SafeTensors:** Prioritize loading models using the `safetensors` format (`.safetensors` files) whenever possible. It is a secure alternative that does not allow for arbitrary code execution. When using `transformers`, load models with `from_pretrained('.../model', safe_serialization=True)`. 2. **Scan Models:** Before loading any model, use scanning tools like `picklescan` to check for malicious payloads within the file. 3. **Isolate Loading Environment:** If you must load a pickled model from an untrusted source, do so within a heavily restricted, sandboxed environment (e.g., a minimal Docker container with no network access or sensitive file mounts) to limit the impact of potential code execution. 4. **Verify Model Source:** Only use models from trusted, verified organizations on platforms like Hugging Face.
Patch Details
This is an insecure design pattern rather than a specific library vulnerability. The patch is procedural: developers must adopt safer serialization formats like `safetensors`.