Malicious Code Execution via Poisoned Pickle-Serialized Models on Hugging Face Hub
Overview
A significant supply chain risk was demonstrated where attackers upload pre-trained models to public repositories like Hugging Face Hub using the insecure `pickle` serialization format (`.pkl` or `.bin` files). The Python pickle module is known to be unsafe for deserializing untrusted data, as it can be instrumented to execute arbitrary code. In this attack, a threat actor creates a malicious `torch.nn.Module` object whose `__reduce__` method is overridden to execute a system command upon deserialization. This malicious object is then saved within the model's weights file. An unsuspecting developer or MLOps pipeline downloads this popular-looking model (e.g., a fine-tuned version of a known architecture) and loads it using `torch.load()`. As soon as the file is loaded, the malicious payload executes, granting the attacker a reverse shell, stealing credentials, or compromising the training environment. This attack is highly effective because loading pre-trained models is a standard practice, and many users are not aware of the underlying risks of the file formats. The `safetensors` format was developed specifically to mitigate this class of vulnerability.
Affected Systems
Testing Guide
1. Download a model file in `.pkl` or `.bin` format from an untrusted source. 2. Before loading it in Python, use a static analysis tool: `pip install picklescan`. 3. Run the scanner on the model file: `picklescan --path /path/to/model.bin`. 4. The tool will flag the file as 'dangerous' if it contains opcodes known to be associated with arbitrary code execution, such as `os.system` or `subprocess.run`. 5. Never load a file that fails this scan in a trusted environment.
Mitigation Steps
1. **Prioritize `safetensors`**: Exclusively download and use models in the `.safetensors` format, which is a secure alternative that does not allow for arbitrary code execution. 2. **Scan Models Before Loading**: Use security scanners like `picklescan` to inspect model files for malicious payloads before deserializing them. 3. **Isolate Loading Process**: Load models from untrusted sources in a sandboxed, network-isolated environment to contain any potential code execution. 4. **Verify Model Source**: Only use models from trusted, verified organizations on platforms like Hugging Face. Check for signs of repository tampering or typosquatting.
Patch Details
This is an inherent risk of the pickle format. The industry mitigation is to adopt the `safetensors` standard, not to patch pickle.