Arbitrary Code Execution via Maliciously Crafted Model Weights on Hugging Face Hub
Overview
A critical vulnerability exists in the common practice of using Python's `pickle` format for serializing and distributing machine learning models. Attackers can craft a malicious model file (e.g., `pytorch_model.bin`) containing a serialized Python object with embedded arbitrary code. When a developer or MLOps pipeline downloads this model from a public repository like the Hugging Face Hub and loads it using standard functions such as `torch.load()` or `pickle.load()`, the deserialization process executes the attacker's code. This can lead to a complete system compromise, providing the attacker with a reverse shell, stealing credentials, or deploying ransomware within the victim's environment. The attack surface is vast, as thousands of developers and organizations download community-provided models daily. While Hugging Face has implemented model scanning and promotes the safer `safetensors` format, the legacy `pickle` format remains widely used, and scanners can be bypassed. This vulnerability underscores the inherent risks in the AI supply chain, where the trust placed in open-source models can be exploited for remote code execution (RCE). The discovery highlighted the urgent need for a shift to secure-by-default serialization formats across the AI ecosystem.
Affected Systems
Testing Guide
1. **Check File Format**: Identify all models used in your applications. Verify if they are stored as `.bin` or `.pt` files (likely pickle) or `.safetensors`. 2. **Run a Scanner**: Download a known malicious proof-of-concept model from a security researcher's repository (in a safe environment) or use a tool like `picklescan` against your existing model directories: ```bash pip install picklescan picklescan --path /path/to/your/models ``` 3. **Review Code**: Search your codebase for instances of `torch.load` and `pickle.load` to identify potential points of ingress for malicious files.
Mitigation Steps
1. **Prefer SafeTensors**: Exclusively use models distributed in the `.safetensors` format. This format does not allow for arbitrary code execution. 2. **Scan Models**: Use tools like Hugging Face's built-in malware scanner or third-party tools like `picklescan` to check model files for dangerous opcodes before loading them. 3. **Sandbox Execution**: Load and run models from untrusted sources in a sandboxed, network-isolated environment (e.g., Docker container with limited permissions) to contain any potential malicious activity. 4. **Code Review Loading Process**: Ensure your code does not use `pickle` with data from untrusted sources. If you must use `pickle`, do not load files you have not personally created or verified.
Patch Details
This is a procedural vulnerability. The primary mitigation is migrating to the `safetensors` format, which is supported in recent versions of major frameworks. Hugging Face has also enhanced its on-platform scanning.