Malicious Model on Hugging Face Hub Executes Remote Code via Pickled Payload
Overview
Researchers from Trail of Bits demonstrated a supply chain attack targeting AI developers using the Hugging Face Hub. A malicious actor uploaded a seemingly benign model, but its weights file (`pytorch_model.bin`) was crafted to exploit the Python `pickle` deserialization mechanism. When a victim loaded the model using the standard `torch.load()` function, which is common practice in many pipelines, the unpickling process executed arbitrary code embedded within the payload. This code established a reverse shell back to the attacker's server, granting them full control over the developer's machine or the production environment where the model was being loaded. The attack highlights a critical trust issue in the open-source model ecosystem, as many developers implicitly trust models from the Hub without thoroughly inspecting their contents or using safer loading formats like `safetensors`. The impact is severe, leading to complete system compromise, data theft, and the potential to poison downstream AI applications. The discovery prompted Hugging Face to enhance its security scanning for known pickle exploits and to more aggressively promote the use of the `safetensors` format, which does not have an arbitrary code execution vulnerability.
Affected Systems
Testing Guide
1. Identify all code paths in your applications that use `torch.load()` or `pickle.load()` to load model files (`.bin`, `.pth`, `.pkl`). 2. Run a static scanner like `picklescan` against your model cache directory (`~/.cache/huggingface/hub` by default). 3. For a dynamic test, create a known-malicious pickle file (e.g., one that writes a file to `/tmp/pwned`) and attempt to load it in a secure, isolated test environment to confirm your security controls (sandboxing, network policies) would block or detect the malicious behavior.
Mitigation Steps
1. **Prioritize Safe Formats:** Always use the `.safetensors` format for model weights when available. It is designed to be safe and does not allow for arbitrary code execution. 2. **Scan Models:** Before loading any third-party model, use security scanners like `picklescan` or Hugging Face's built-in malware scanner to detect malicious payloads. 3. **Isolate Loading:** Load untrusted models in a sandboxed, network-isolated environment (e.g., a container with no network access) to prevent reverse shells or data exfiltration if a payload executes. 4. **Update Libraries:** Ensure PyTorch and Hugging Face Transformers libraries are updated to versions that include enhanced warnings and security features regarding unsafe deserialization.
Patch Details
No direct patch for the pickle vulnerability exists as it's intended behavior. Mitigation relies on using `safetensors` and updating Hugging Face Transformers to >= 4.41.0 for improved scanning and warnings.