Malicious Model Execution on Hugging Face Hub via Unsafe Pickle Deserialization
Overview
A widespread supply chain vulnerability was demonstrated by security firm Trail of Bits, affecting the broader machine learning ecosystem centered around the Hugging Face Hub. The research revealed that a significant number of publicly available models on the Hub could lead to arbitrary code execution on a user's machine during the model loading process. This is not a flaw in the Hub itself, but rather an insecure practice in the ML community. Many models, particularly those in the PyTorch format (`.pth` or `.bin`), use Python's `pickle` module for serialization. The `pickle` format is notoriously unsafe as it can be crafted to execute arbitrary Python code upon deserialization. An attacker can upload a seemingly benign model to the Hub, but embed a malicious payload within its pickled data. When a victim downloads and loads this model using a standard command like `torch.load('model.pth')`, the payload executes, giving the attacker full control over the victim's machine. The research team identified dozens of models with active payloads, ranging from simple reverse shells to more sophisticated infostealers. This discovery forced a reckoning within the MLOps community, accelerating the adoption of safer serialization formats like `safetensors`.
Affected Systems
Testing Guide
1. **Identify Models Using Pickle:** Check if your projects load models from `.pth`, `.pkl`, or `.bin` files from untrusted sources on the Hub. 2. **Use a Scanning Tool:** Install `picklescan` via `pip install picklescan`. 3. **Scan a Model File:** Run the scanner on a model file you intend to use: `picklescan -p /path/to/model.pth`. 4. **Review Findings:** The tool will report if it finds any imports or calls that are indicative of malicious activity. A finding of `__builtin__.exec` or `os.system` is a critical red flag.
Mitigation Steps
1. **Use Safe Serialization Formats:** Prioritize loading models using the `.safetensors` format, which is designed to be secure and prevent arbitrary code execution. Use `from_safetensors=True` where available. 2. **Scan Models Before Use:** Employ model scanning tools like Hugging Face's built-in malware scanner or third-party tools like `picklescan` to check for suspicious operators in model files before loading them. 3. **Load from Trusted Sources:** Only download and use models from verified and reputable publishers on the Hugging Face Hub. 4. **Isolate Loading Process:** If you must use a `.pth` file from an untrusted source, load it in a sandboxed, network-isolated environment to contain any potential malicious activity.
Patch Details
This is an ecosystem-wide issue, not a specific software bug. Hugging Face has implemented server-side scanning and promotes the 'safetensors' format as the secure default.