Arbitrary Code Execution via Maliciously Crafted Models on Hugging Face Hub
Overview
A significant supply chain risk was highlighted by security researchers demonstrating how models hosted on the Hugging Face Hub could be weaponized to achieve arbitrary code execution on a victim's machine. The attack vector relies on models saved using Python's `pickle` serialization format, which is notoriously insecure as it can execute arbitrary code upon deserialization. An attacker can create a custom PyTorch model class with a malicious `__reduce__` method. This method, which is called during unpickling, can be engineered to execute system commands. The attacker then saves an instance of this malicious model, generating a `pytorch_model.bin` file, and uploads it to the Hugging Face Hub, often masquerading as a popular or useful model. When an unsuspecting user downloads this model and loads it using the standard `transformers` library pipeline (e.g., `AutoModel.from_pretrained(...)`), the `pickle.load()` function is invoked internally, triggering the RCE payload. This attack is highly potent as it subverts the trust developers place in the open-source model ecosystem. The widespread adoption of the `safetensors` format is a direct mitigation, but thousands of legacy models using `pickle` remain on the Hub, posing an ongoing threat.
Affected Systems
Testing Guide
1. **DO NOT EXECUTE ON A PRODUCTION MACHINE.** Use a sandboxed environment. 2. Find a model on the Hugging Face Hub that only contains a `pytorch_model.bin` file and no `.safetensors` equivalent. 3. Before loading, inspect the repository for any suspicious files or code. 4. Attempt to load the model using a script. A malicious model could trigger unexpected network connections or file system modifications. A safe way to test is to use a pickle scanner on the downloaded `.bin` file to detect malicious opcodes without executing the payload.
Mitigation Steps
1. **Prioritize SafeTensors**: Only load models that use the `.safetensors` format. Check the 'Files and versions' tab on the model repository page. 2. **Scan Models**: Before loading any model, use a tool like `picklescan` to check for dangerous opcodes in `.bin` files. 3. **Enable Safe Loading**: When using `from_pretrained`, consider using the `safe_serialization=True` argument if available in your library version, which forces the use of safetensors. 4. **Distrust Unknown Models**: Avoid using models from unknown or untrusted publishers without thorough vetting.
Patch Details
This is a systemic risk with the pickle format. Mitigation relies on ecosystem-wide adoption of the safer `safetensors` format, which is promoted by Hugging Face but not enforced for all models.