Malicious Code Execution via Poisoned Pickle Serialization in Community-Uploaded Models on Hugging Face Hub
Overview
Security researchers from ProtectAI uncovered a targeted supply chain attack involving over 50 malicious models uploaded to the Hugging Face Hub. The attackers forked popular, legitimate open-source models like 'mistralai/Mistral-7B-Instruct-v0.3', then replaced the standard `pytorch_model.bin` file with a malicious version. This file was a serialized Python object using the insecure `pickle` module. When an unsuspecting user would load the model using the standard `AutoModelForCausalLM.from_pretrained()` function from the `transformers` library, the `pickle.load()` operation would execute an embedded `__reduce__` method. This method contained Python code to download and execute a second-stage payload, typically a cryptominer or an information-stealing trojan that harvested SSH keys and cloud credentials. The attack exploited the implicit trust developers place in the model hub and the inherent dangers of deserializing untrusted data with `pickle`. The attack specifically targeted developers' machines and CI/CD environments where models are frequently downloaded and tested, aiming to steal secrets. Hugging Face has since implemented enhanced scanning for `pickle` archives and now displays prominent warnings on model pages that use this format.
Affected Systems
Testing Guide
1. Identify a model on the Hugging Face Hub that only provides a `pytorch_model.bin` file. 2. Before loading it, run a security scanner on the file: `pip install picklescan && picklescan -p /path/to/pytorch_model.bin`. 3. If the scanner reports a `dangerous-code` finding, the model is potentially malicious. 4. Do not load any model that is flagged by the scanner in a production or sensitive environment.
Mitigation Steps
1. Prioritize using models that are available in the `safetensors` format, which does not allow for arbitrary code execution. 2. When loading a `pickle`-formatted model, never use `trust_remote_code=True` unless you have manually audited the model's source code. 3. Before loading any community-provided model, use a static analysis tool like `picklescan` to scan the `pytorch_model.bin` file for malicious payloads. 4. Upgrade the `transformers` library to a version (>=4.40.0) that displays explicit warnings when loading pickle-serialized models.
Patch Details
This is an attack pattern, not a specific software bug. Mitigation relies on user awareness and tooling. Hugging Face has improved platform warnings, and libraries have added more explicit user prompts.