Remote Code Execution via Malicious Pickle Deserialization in Community-Uploaded Hugging Face Models
Overview
A persistent supply chain threat within the AI ecosystem involves the use of Python's `pickle` serialization format for distributing machine learning models. Security firm ProtectAI discovered a malicious campaign where threat actors uploaded over a dozen models to the Hugging Face Hub, masquerading as popular fine-tuned versions of models like Llama 3 or Stable Diffusion. These models' `pytorch_model.bin` files were not standard model weights but rather malicious pickle payloads. When a developer or researcher downloads and loads such a model using common code patterns like `torch.load('pytorch_model.bin')`, the `pickle.load()` function is called under the hood. Pickle is not secure against erroneous or maliciously constructed data; deserializing a pickle stream can execute arbitrary code. In this campaign, the payload established a reverse shell connection to an attacker-controlled server, granting them access to the user's development environment. This access could be used to steal sensitive data, such as API keys, proprietary code, or cloud credentials. The incident highlights the critical need to move away from pickle-based model formats and adopt safer alternatives like `safetensors`, which only allows for the loading of tensor data and does not permit arbitrary code execution. Hugging Face has implemented scanning and warnings, but the ultimate responsibility lies with the user to validate the source and format of downloaded models.
Affected Systems
Testing Guide
1. **Obtain a Model File:** Download a model file (`.bin`, `.pth`) that you wish to test from an untrusted source. 2. **Install Scanner:** Install a pickle scanning tool: `pip install picklescan`. 3. **Scan the File:** Run the scanner on the model file from your terminal: `picklescan -p /path/to/your/model.bin`. 4. **Review Results:** The tool will report if it finds any suspicious global imports or reduce functions that could lead to code execution. If any are found, do not load the file.
Mitigation Steps
1. **Prefer SafeTensors:** Prioritize loading models in the `.safetensors` format whenever possible. When using Hugging Face's `from_pretrained` method, it will automatically prefer safetensors if available. 2. **Scan Model Files:** Before loading any model weights, especially in pickle format, scan them with a tool like `picklescan` to detect potentially malicious code. 3. **Verify Model Source:** Only use models from trusted, verified creators on platforms like Hugging Face Hub. Be wary of newly uploaded models or those from unknown authors. 4. **Disable Arbitrary Code Execution:** When using `torch.load`, ensure that you trust the source completely. For models from untrusted sources, load them in a sandboxed, network-isolated environment to contain any potential malicious activity.
Patch Details
This is a procedural vulnerability. Mitigation relies on user behavior and tooling, not a direct software patch.