Malicious Pickle Deserialization in Hugging Face Hub Models Leading to Remote Code Execution
Overview
A widespread security incident highlighted the long-standing risks of using Python's `pickle` format for model serialization. Attackers uploaded several seemingly benign models to the Hugging Face Hub, targeting popular architectures like BERT and LLaMA. These models' `pytorch_model.bin` files, which are serialized using `pickle`, were modified to contain a malicious payload. The payload was embedded within a custom class definition whose `__reduce__` method was overridden to execute arbitrary system commands via `os.system()`. When an unsuspecting developer or MLOps pipeline loaded one of these models using the standard `transformers.AutoModel.from_pretrained('malicious/model')` call, the `pickle.load()` operation would trigger the payload. This resulted in immediate code execution on the user's machine, leading to stolen SSH keys, cloud credentials (`~/.aws/credentials`), and API keys from environment variables. The incident was discovered after multiple users on the Hugging Face forums reported their cloud accounts being compromised shortly after using a newly published, highly-rated community model. This event forced a renewed push for the adoption of the safer `safetensors` format and led to Hugging Face implementing more stringent scanning and warnings for models using the pickle format.
Affected Systems
Testing Guide
1. **Install a Scanner:** Run `pip install picklescan`. 2. **Download a Model:** Use `huggingface-cli` to download a model repository without loading it in Python. 3. **Scan the Files:** Run `picklescan -p /path/to/model/pytorch_model.bin`. 4. **Review Results:** The scanner will report if it finds potentially malicious global imports like `os.system` or `subprocess.run`. A finding indicates a high risk.
Mitigation Steps
1. **Prefer SafeTensors:** Explicitly load models using the `safetensors` format by default. When using `from_pretrained`, set `use_safetensors=True` where available. 2. **Scan Models Before Use:** Use tools like `picklescan` to scan model files for dangerous opcodes before loading them. 3. **Disable Arbitrary Code Execution:** Load models from trusted sources only. For untrusted models, use a sandbox or container to limit the potential impact of RCE. 4. **Audit Dependencies:** Ensure that you are loading models from the official repositories of trusted organizations on the Hub.
Patch Details
This is an inherent risk of the pickle format. Mitigation relies on user awareness and tooling like safetensors, not a direct patch to the format itself. Hugging Face improved platform-side scanning and warnings.