Remote Code Execution in Hugging Face Hub via Malicious Pickled Model Weights
Overview
A supply chain vulnerability was identified where malicious actors could upload a seemingly benign machine learning model to the Hugging Face Hub. The vulnerability exploits the insecure deserialization of Python objects when using the `pickle` format, which is a common method for saving and loading PyTorch models (`pytorch_model.bin`). An attacker crafts a model file containing a malicious payload within its pickled data. When a developer or an automated MLOps pipeline downloads and loads this model using `torch.load()`, the deserialization process executes arbitrary code embedded by the attacker. This can lead to a complete compromise of the machine loading the model, whether it's a developer's laptop, a training server, or a production inference endpoint. The impact is critical, allowing for remote code execution (RCE), data theft, or lateral movement within the victim's network. This attack vector highlights the significant trust placed in open-source model repositories and the inherent dangers of insecure file formats in the AI ecosystem. The discovery was made by security researchers who demonstrated the ease of creating and distributing such a 'model bomb'.
Affected Systems
Testing Guide
1. **Identify Usage of `pickle`**: Search your codebase for calls to `torch.load()` or `pickle.load()` that operate on model files downloaded from the internet. 2. **Use a Scanning Tool**: Run a tool like `picklescan` against your local model cache directory (e.g., `~/.cache/huggingface/hub`). Example command: `picklescan -p /path/to/your/models`. 3. **Check for Malicious Payloads**: The scanner will flag any files containing opcodes commonly used for RCE, such as `os.system` or `subprocess.run`. 4. **Review Loading Logic**: Confirm if your application logic enforces loading from `safetensors` files by default (`from_pretrained(..., use_safetensors=True)`).
Mitigation Steps
1. **Prioritize Safe Tensors**: Always prefer loading models using the `safetensors` format (`.safetensors` file extension) when available. It is a secure alternative to `pickle` that does not allow for arbitrary code execution. 2. **Scan Models Before Use**: Employ security scanners like `picklescan` or commercial ML security platforms to analyze model files for malicious payloads before loading them. 3. **Isolate Model Loading**: Execute model loading processes in sandboxed, isolated environments (e.g., containers with minimal privileges, no network access) to limit the potential impact of a compromise. 4. **Vet Model Sources**: Only use models from trusted, verified organizations on platforms like Hugging Face. Be cautious with newly uploaded models or those from unknown publishers.
Patch Details
This is a procedural vulnerability in the ecosystem rather than a specific software bug. The primary fix is adopting the `safetensors` format and secure practices.