Arbitrary Code Execution via Malicious Models on Hugging Face Hub
Overview
A persistent supply chain attack vector was demonstrated where malicious actors upload seemingly legitimate pre-trained models to the Hugging Face Hub that contain arbitrary code execution payloads. The attack leverages the common practice of serializing model weights and configurations using Python's `pickle` format, which is known to be unsafe when deserializing data from untrusted sources. Many popular frameworks, including PyTorch, use `pickle` under the hood. An attacker can craft a `pytorch_model.bin` or `tf_model.h5` file that, when loaded by a victim using standard library functions like `torch.load()`, executes embedded Python bytecode. This payload can perform a wide range of malicious actions, from installing malware and exfiltrating data (including API keys and user data) to establishing a persistent reverse shell on the researcher's or company's machine. The attack is particularly insidious because it requires no vulnerability in the Hugging Face platform or the ML frameworks themselves, but rather exploits the trust users place in the open-source model ecosystem. The Hugging Face team has implemented scanning for known malicious payloads, but novel obfuscation techniques can bypass these checks, making user vigilance a critical line of defense.
Affected Systems
Testing Guide
1. **Set up a Test Environment**: Create a firewalled virtual machine or container. 2. **Download a Test Model**: Download a model file from an unverified source (or use a security research tool like `malicious-model-maker` to create a safe test case). 3. **Run a Scanner**: Use a tool like `picklescan` on the model files (e.g., `picklescan -p /path/to/model/pytorch_model.bin`). 4. **Check for Findings**: If the scanner flags the file as containing potentially dangerous opcodes, the model loading process would be unsafe.
Mitigation Steps
1. **Enable Safetensors**: Prefer loading models using the `.safetensors` format, which is a secure alternative to pickle that does not allow for arbitrary code execution. Check if a model has a `safetensors` version available. 2. **Scan Models Before Use**: Use tools like `picklescan` or other open-source scanners to inspect model files for malicious payloads before loading them into memory. 3. **Isolate Model Loading**: Load and run models from untrusted sources in a sandboxed, network-isolated environment to limit the potential impact of a compromise. 4. **Verify Model Source**: Only use models from trusted and verified organizations or authors on the Hub. Check for signs of legitimacy like high download counts, organizational verification, and community discussion.
Patch Details
This is a procedural risk, not a specific software vulnerability. Mitigation relies on user awareness and secure practices, such as adopting the safetensors format.