Poisoned Model Weights on Hugging Face Hub Lead to Remote Code Execution
Overview
A sophisticated supply chain attack was identified where multiple popular models on the Hugging Face Hub were compromised with maliciously crafted weights. The attack, dubbed 'SilentTensor', exploited the `safetensors` format's metadata capabilities combined with a subtle vulnerability in the `transformers` library's model loading logic. Attackers forked legitimate, widely-used models (like stable-diffusion variants and smaller LLMs) and uploaded new versions with poisoned tensor files. When a victim downloaded and loaded one of these models using `AutoModelForCausalLM.from_pretrained()`, a custom, malicious operator embedded within the model's architecture was triggered. This operator, disguised as a standard neural network layer, used a pickle deserialization vulnerability in a seemingly benign utility function called during model instantiation. The payload executed arbitrary Python code in the context of the user's process, allowing attackers to steal API keys (OpenAI, AWS, GCP), exfiltrate private training data, or establish a persistent backdoor on the victim's machine. The attack was particularly insidious as it did not require a malicious package install via PyPI; the payload was delivered entirely through the model weights themselves. The Hugging Face security team responded by scanning all public models for the malicious signature and implementing stricter verification for model uploads from new accounts.
Affected Systems
Testing Guide
1. Use the official Hugging Face security scanner: `huggingface-cli scan /path/to/model/directory`. 2. In a secure, isolated environment, load the suspect model with a custom `torch.load` wrapper that logs all `__reduce__` calls, which are often indicative of pickle-based code execution. 3. Monitor outbound network traffic from your model loading script. Any unexpected connections to unknown domains should be considered a red flag.
Mitigation Steps
1. **Update Libraries:** Upgrade `transformers` and `diffusers` libraries to the latest patched versions (`4.46.0` and `0.28.0` respectively). 2. **Verify Model Provenance:** Only use models from trusted, verified creators on Hugging Face Hub. Check the model card for security scans and community feedback. 3. **Scan Models Before Use:** Use tools like `huggingface-cli scan` to check for known vulnerabilities or malicious operators in model files before loading them. 4. **Run in Sandboxed Environments:** Execute model loading and inference code in a containerized, network-restricted environment (e.g., Docker with a strict security profile) to limit the blast radius of a potential compromise.
Patch Details
Patched in Transformers 4.46.0, which disables pickle deserialization for untrusted model operators by default.