Arbitrary Code Execution via Malicious Pickle Deserialization in Hugging Face Models
Overview
Research revealed a critical supply chain vulnerability affecting the Hugging Face ecosystem, allowing attackers to achieve remote code execution (RCE) on machines that download and load seemingly safe models. The attack vector abuses the inherent risks of Python's `pickle` serialization format. While the `.safetensors` format was introduced as a secure alternative to `pickle` for storing model weights, it was discovered that the `transformers` library's loading mechanism could still insecurely load a malicious `data.pkl` file if it was included within a `.safetensors` model's repository. Attackers could craft a model, embed a malicious pickle payload containing arbitrary code, and upload it to the public Hugging Face Hub. When an unsuspecting developer or MLOps pipeline downloads and loads this model using standard library functions like `AutoModel.from_pretrained()`, the malicious pickle payload is deserialized, triggering the RCE. This vulnerability circumvents the security promises of the `.safetensors` format and highlights the complexity of securing the AI supply chain, where trust is often implicitly placed in public model repositories. The impact is severe, as it allows for complete compromise of developer workstations and production AI inference servers.
Affected Systems
Testing Guide
1. **Obtain a Scanner**: Install a pickle scanner: `pip install picklescan`. 2. **Scan Model Directory**: Before loading a model from an untrusted source, run the scanner on its local directory: `picklescan -p /path/to/model/files`. 3. **Analyze Results**: If the scanner reports findings of `dangerous_global` imports (e.g., `os.system`), the model is likely malicious and should not be loaded.
Mitigation Steps
1. **Upgrade Libraries**: Update `huggingface-hub` to version `0.20.0` or later and `transformers` to `4.36.0` or later. Patched versions introduce stricter checks and warnings for potentially unsafe model files. 2. **Scan Models Before Use**: Use security scanners like `picklescan` or `mreduce` to inspect any third-party model files for embedded pickle payloads before loading them. 3. **Use Trusted Publishers**: Only download and use models from verified creators and organizations on the Hugging Face Hub. 4. **Restrict Permissions**: Run model loading and inference processes in low-privilege, sandboxed environments to limit the potential impact of a compromise.
Patch Details
Upgrade `huggingface-hub` to 0.20.0+ and `transformers` to 4.36.0+. The patch introduces safer loading mechanisms and warnings for untrusted files.