Model Poisoning via Malicious Pickle Deserialization in PyTorch Models
Overview
A persistent supply chain vulnerability affects AI developers who download pre-trained models from public repositories like Hugging Face Hub. The vulnerability exists because many model formats, including PyTorch's legacy `.bin` or `.pt` files, use Python's `pickle` serialization format to store model weights and architecture. The `pickle` module is notoriously insecure as it can be used to execute arbitrary code during deserialization (`loading`). An attacker can craft a malicious model file where one of the serialized objects is a custom class with a `__reduce__` method that executes a system command. They then upload this poisoned model to a public hub, often disguised as a popular, legitimate model. When a victim downloads and loads the model using `torch.load('malicious_model.pt')`, the embedded code executes on their machine with the user's privileges. This can lead to credential theft, installation of ransomware, or the establishment of a persistent backdoor on the developer's machine or in a production environment. This attack vector highlights the critical need for secure model serialization formats and for developers to treat model weights as untrusted, executable code.
Affected Systems
Testing Guide
1. **Install a scanner**: `pip install picklescan` 2. **Download a model**: Obtain a model file (`.bin`, `.pt`, `.pth`) that you wish to test. 3. **Run the scanner**: Execute `picklescan --path /path/to/your/model.bin` from your terminal. 4. **Analyze Results**: The tool will report if it finds any suspicious global imports or opcodes that could indicate a malicious pickle payload. A safe model will return a 'No infected files found' message.
Mitigation Steps
1. **Use Safe Tensors**: Exclusively use models saved in the `safetensors` format, which is designed to be secure and does not have an arbitrary code execution vector. Many models on Hugging Face Hub have a `safetensors` version available. 2. **Scan Models**: Before loading any model, use tools like `picklescan` to scan for malicious payloads in the files. 3. **Isolate Loading**: If you must load a pickled model from an untrusted source, do so in a heavily sandboxed, network-isolated environment to contain any potential code execution. 4. **Trust Established Sources**: Only download models from highly reputable organizations and publishers, and verify their authenticity.
Patch Details
This is a fundamental risk of the pickle format. The mitigation is to avoid it entirely by using safer formats like safetensors.