Backdoored 'SafeTensor' Models on Hugging Face Hub Execute Malicious Code on Load
Overview
Security researchers at the AI Village discovered a novel supply chain attack vector targeting models hosted on the Hugging Face Hub. The attack embeds a malicious payload within a model's repository that executes during the model loading process, even when using the supposedly safer `.safetensors` format. The attack circumvents the inherent safety of SafeTensors by abusing the `trust_remote_code=True` argument in the Hugging Face `transformers` library. Attackers upload a seemingly legitimate fine-tuned model to the Hub. However, the model's repository also contains a malicious Python script (e.g., within the model's configuration or a utility script). When a developer or an MLOps pipeline loads this model using the common `AutoModel.from_pretrained('attacker/malicious-model', trust_remote_code=True)` pattern, the library executes the attacker's code. This provides the attacker with remote code execution capabilities on the victim's machine, allowing for theft of credentials, environment variables, proprietary data, or lateral movement within the network. This incident highlights the critical importance of vetting third-party models and the significant risk associated with trusting remote code, a feature often required for custom model architectures. The researchers demonstrated that thousands of public models on the Hub require this flag, creating a large attack surface for unsuspecting developers.
Affected Systems
Testing Guide
1. Identify all projects where you load models from the Hugging Face Hub. 2. Search your codebase for the string `trust_remote_code=True`. 3. For each occurrence, identify the source of the model being loaded (e.g., `meta-llama/Llama-2-7b-chat-hf` vs. `some-new-user/cool-model-v1`). 4. If the model is from an unverified or unknown publisher, consider yourself potentially at risk. Immediately review the model's repository on the Hugging Face Hub for any suspicious scripts.
Mitigation Steps
1. **Never use `trust_remote_code=True`** with models from untrusted sources. Scrutinize any model that requires this flag. 2. Before loading a model, manually inspect all files in its Hugging Face repository, particularly any Python scripts (`.py` files), for suspicious code. 3. Use model scanning tools like `safetensors-check` or other community tools to analyze model files for known malicious patterns before loading them. 4. Run model loading and inference in a sandboxed, isolated environment (e.g., a container with no network access and minimal permissions) to limit the potential impact of a compromise.
Patch Details
This is an attack pattern exploiting a feature's intended behavior. Mitigation relies on user security practices, not a direct patch.