Malicious Code Execution via Pickled Model Weights on Hugging Face Hub
Overview
Researchers demonstrated a supply chain attack vector where a maliciously crafted model on the Hugging Face Hub can lead to arbitrary code execution on a victim's machine during the model loading process. The attack leverages the common practice of using Python's `pickle` format for serializing model weights and components. The `pickle` module is known to be insecure against maliciously constructed data, as it can be used to execute arbitrary code during deserialization. The attack involves creating a seemingly benign model (e.g., a fine-tuned version of a popular LLM) but embedding a malicious payload within one of its `pytorch_model.bin` files. When a developer or MLOps pipeline downloads and loads this model using standard library functions like `torch.load()` or a high-level API from the `transformers` library that uses `pickle` under the hood, the embedded payload is executed. This provides the attacker with a foothold on the machine, which could be a developer's laptop or a production inference server. The payload can be used to steal credentials, exfiltrate data, or install persistent malware. The attack is particularly insidious because it requires no user interaction beyond the standard model loading procedure, and security scanners may not detect the payload within the binary model weights. This underscores the risk of trusting community-contributed models without proper vetting and sandboxing.
Affected Systems
Testing Guide
1. Review your MLOps pipelines and development scripts. Identify all instances where models are loaded from external sources. 2. Check if the loading mechanism uses `pickle` implicitly or explicitly (e.g., `torch.load` on a `.bin` file). 3. Attempt to load a known malicious model file (a "pickle bomb" EICAR-like file) in a controlled, isolated test environment. 4. Observe if arbitrary code is executed (e.g., a file is created, a network connection is made). If so, your loading process is vulnerable. 5. Switch your code to preferentially load `.safetensors` files and verify that the same malicious `.bin` file is rejected or fails to load.
Mitigation Steps
1. **Use `safetensors`.** Whenever possible, exclusively load models that are distributed in the `.safetensors` format, which is designed to be a secure alternative to `pickle`. 2. **Scan models for malicious code.** Use tools like `huggingface-cli scan` to check models for unsafe modules or pickle imports before loading them. 3. **Isolate model loading.** Perform model loading and inference within a sandboxed, network-isolated environment (e.g., a minimal Docker container) to limit the blast radius of a potential compromise. 4. **Vet model sources.** Only use models from trusted, verified creators on platforms like Hugging Face. Be highly suspicious of newly uploaded models from unknown authors.
Patch Details
This is an inherent risk of the pickle format. Mitigation involves using the `safetensors` format. Hugging Face now defaults to and promotes `safetensors`.