Remote Code Execution via Malicious 'pickle' Serialized Models on Hugging Face Hub
Overview
A persistent supply chain risk within the AI ecosystem involves the use of Python's `pickle` format for serializing and sharing machine learning models. The Hugging Face Hub, a central repository for AI models, hosts thousands of models that use this format. The `pickle` module is not secure against erroneously or maliciously constructed data; deserializing a pickled object can execute arbitrary code. An attacker can create a malicious model file (e.g., `pytorch_model.bin`) containing a pickled object with an embedded payload. This is achieved by defining a class with a `__reduce__` method that, when unpickled, calls a function like `os.system()` with attacker-controlled arguments. The attacker then uploads this poisoned model to the Hugging Face Hub, often disguised as a legitimate or fine-tuned version of a popular model. An unsuspecting developer or MLOps pipeline that downloads and loads this model using standard library functions like `torch.load()` or `pickle.load()` would trigger the payload, leading to remote code execution on their machine or production server. This attack vector is particularly dangerous because the malicious code is hidden within binary model weights, bypassing typical source code scanners. The impact is critical, granting the attacker a foothold in development environments or production AI infrastructure, potentially leading to data theft, model poisoning, or ransomware deployment. The ML community's response has been to promote safer serialization formats.
Affected Systems
Testing Guide
1. Create a malicious pickle file. A simple Python script can do this: `import pickle; import os class RCE: def __reduce__(self): return (os.system, ('echo "VULNERABLE"',)) with open('malicious.pkl', 'wb') as f: pickle.dump(RCE(), f)` 2. In a separate, isolated test environment, attempt to load the file: `import pickle; pickle.load(open('malicious.pkl', 'rb'))` 3. If the string "VULNERABLE" is printed to the console, the environment's deserialization process is unsafe and would execute a more malicious payload.
Mitigation Steps
1. **Use SafeTensors**: Prioritize loading models exclusively in the `safetensors` format. This format is designed for safety and does not allow for arbitrary code execution. 2. **Scan Models**: Use model scanning tools like Hugging Face's `hf-hub-scanner` or other third-party solutions to check for suspicious pickle imports before loading a model. 3. **Sandbox Loading**: If you must load a pickle file from an untrusted source, do so in a heavily isolated and sandboxed environment with no network access or sensitive data. 4. **Code Review**: When using custom models, review the model's source code for any custom class definitions that might contain malicious `__reduce__` methods.
Patch Details
This is an architectural risk associated with the pickle format, not a bug that can be patched in a single library. The industry-wide mitigation is to migrate to the `safetensors` format.