Arbitrary Code Execution via Maliciously Crafted Model on Hugging Face Hub
Overview
Security researchers discovered a critical supply chain vulnerability affecting developers who download pre-trained models from the Hugging Face Hub. The attack vector involves a malicious actor uploading a seemingly benign model to the Hub. However, the model is saved using Python's `pickle` serialization format. The `pickle` format is known to be insecure as it can execute arbitrary code upon deserialization. The attacker crafts a malicious `pickle` file containing a payload that executes when a developer loads the model using standard library functions like `torch.load()` or by using the Hugging Face `transformers` library without explicitly enabling `safetensors`. The payload in the proof-of-concept established a persistent backdoor on the developer's machine, stole cloud credentials from environment variables (AWS, GCP, Azure), and exfiltrated proprietary training data. This attack highlights a significant blind spot in the MLOps pipeline, where model files are often treated as trusted data assets. The incident prompted Hugging Face to enhance its security scanning for public models and more aggressively promote the use of the `safetensors` format, which is a secure alternative for storing model weights that does not allow for arbitrary code execution.
Affected Systems
Testing Guide
1. **Inventory Models**: Identify all `.pkl`, `.pt`, and `.bin` model files used in your projects that are sourced from public repositories. 2. **Static Analysis**: Run a pickle security scanner on these files. For example: `fickling --check <path-to-model-file>`. The tool will flag any files containing potentially dangerous opcodes. 3. **Review Loading Code**: Audit your codebase for calls to `torch.load()`, `pickle.load()`, and `AutoModel.from_pretrained()`. Ensure that `safetensors` is preferred and that models are not loaded from untrusted `pickle` files.
Mitigation Steps
1. **Use SafeTensors**: Exclusively use models in the `safetensors` format. When loading models, explicitly pass the `use_safetensors=True` argument where available. 2. **Scan Models**: Before loading any model from an untrusted source, use a pickle scanner tool like `pysafebrowsing` or `fickling` to inspect the file for malicious opcodes. 3. **Isolate Model Loading**: Perform model loading and initial processing in an isolated, short-lived environment (e.g., a minimal Docker container with no network access or secrets) to limit the blast radius of a potential compromise. 4. **Vet Model Sources**: Only use models from trusted, verified organizations on Hugging Face Hub. Check the model card for details on its origin and training process.