Malicious Model Execution via Unsafe Deserialization in Hugging Face Transformers
Overview
A widespread supply chain risk affects AI developers using popular model hubs like Hugging Face. The vulnerability is not in the platform itself but in the file formats and loading mechanisms commonly used. Many models are saved using Python's `pickle` format, which is known to be insecure as it can execute arbitrary code during deserialization. Attackers create and upload seemingly legitimate models to public repositories. When a victim downloads and loads the model using standard library functions like `torch.load()` on a `.pth` file, a malicious payload embedded within the pickle data is executed. This provides the attacker with a remote shell on the developer's machine or production server. This technique was demonstrated in the 'laser-donkey' incident, where a proof-of-concept malicious model was uploaded to Hugging Face. The impact is severe, leading to data theft, lateral movement within a network, or the compromise of ML training infrastructure. This attack vector highlights the critical need for a secure model serialization format and developer awareness.
Affected Systems
Testing Guide
1. **Audit Dependencies**: Check your `requirements.txt` or `pyproject.toml` for ML libraries like `torch` and `transformers`. 2. **Review Model Loading Code**: Search your codebase for calls to `torch.load()` or `pickle.load()`. Verify that these are only used on files you trust completely. 3. **Check File Formats**: In your model asset directories, check if you are using `.pth`, `.pt`, or `.bin` files from untrusted sources. Prefer `.safetensors`. 4. **Run a Scanner**: Use a tool like `picklescan` against your model directory: `picklescan -p /path/to/your/models` to check for known malicious payloads.
Mitigation Steps
1. **Use SafeTensors**: Prioritize using models in the `.safetensors` format. This format is designed for safety and does not have code execution capabilities. Use `from_pretrained(..., use_safetensors=True)` in Hugging Face. 2. **Scan Models**: Use security scanners like `picklescan` or other vulnerability scanners on any downloaded model files before loading them. 3. **Verify Model Source**: Only download models from trusted, verified organizations and authors on platforms like Hugging Face. 4. **Isolate Loading Process**: If you must load a pickled file from an untrusted source, do so in a sandboxed, network-isolated environment to contain any potential malicious activity.
Patch Details
This is a design-level risk in the ecosystem. The mitigation is to adopt safer formats like SafeTensors, which is now the default for new models on Hugging Face.