Malicious Model on Hugging Face Hub Leads to Remote Code Execution via Unsafe Deserialization
Overview
Researchers at Trail of Bits demonstrated a supply chain attack targeting developers who download pre-trained models from the Hugging Face Hub. The attack leverages the common practice of using Python's `pickle` format to save and load model weights, particularly within PyTorch (`.pth` or `.bin` files). The `pickle` module is known to be unsafe as it can execute arbitrary code during deserialization. The attack involves uploading a seemingly legitimate model to the Hub, but with a malicious payload embedded in one of the model's files. When an unsuspecting developer or an automated MLOps pipeline downloads and loads the model using standard `torch.load()`, the payload executes. The researchers crafted a proof-of-concept that established a reverse shell back to an attacker-controlled server upon model loading, granting them full control over the victim's machine. This highlights a significant supply chain risk in the AI ecosystem, where trust is often implicitly placed in community-contributed models. The attack bypasses static analysis tools, as the payload is hidden within the binary model weights file and is only activated at runtime.
Affected Systems
Testing Guide
1. **Check Model Format:** Review your MLOps pipelines and development workflows. Identify any instances where `torch.load()` or `pickle.load()` are used to load `.pth`, `.bin`, or `.pkl` files from untrusted sources. 2. **Use a Scanning Tool:** Run a model security scanner on the models you have downloaded. The scanner should check for suspicious opcodes within pickle-serialized files. 3. **Safe Loading Test:** Attempt to load a suspect model using a library that explicitly disallows pickle, such as `safetensors.torch.load_file()`. If it fails, the model likely relies on pickle serialization and carries a higher risk.
Mitigation Steps
1. **Use SafeTensors:** Prioritize loading models exclusively in the `safetensors` format, which is designed to be secure against arbitrary code execution. Convert models if necessary. 2. **Scan Models:** Use tools like Hugging Face's built-in malware scanner or third-party model scanning solutions to check for malicious code before loading. 3. **Verify Model Source:** Only use models from trusted, verified organizations and authors on the Hub. Scrutinize models with low download counts or recent uploads from unknown publishers. 4. **Isolate Loading Environment:** Load and process new models in a sandboxed, network-isolated environment to contain any potential malicious activity.
Patch Details
This is an ecosystem-wide risk rather than a specific software bug. Mitigation relies on user awareness and tooling. PyTorch and Hugging Face have improved documentation and warnings regarding this vector.