Remote Code Execution via Maliciously Crafted Model on Hugging Face Hub
Overview
A significant supply chain risk was demonstrated where attackers can upload a maliciously crafted AI model to the Hugging Face Hub, leading to remote code execution (RCE) on the victim's machine. The primary attack vector involves models saved in the legacy `pickle` format (`.bin` files). Python's `pickle` module is known to be insecure for deserializing untrusted data, as it can be engineered to execute arbitrary code. An attacker can embed a malicious payload within a model's layer, which gets executed when a user loads the model using `torch.load()`. While the community has shifted towards the safer `safetensors` format, a secondary vector remains. Attackers can still achieve RCE by placing malicious code in a repository's `__init__.py` or other Python files and tricking users into loading the model with the `trust_remote_code=True` flag in the `from_pretrained()` method. This flag is often required for custom architectures, making it a tempting option for users. Once executed, the malicious code runs with the full permissions of the user, allowing it to steal credentials, install malware, or compromise the development/production environment.
Affected Systems
Testing Guide
1. In your project's codebase, search for all instances of `from_pretrained`. 2. Check if any of these calls include the argument `trust_remote_code=True`. 3. If found, verify that the model being loaded is from a highly trusted and verified creator (e.g., Google, Meta, MistralAI). 4. If the source is not a major, trusted organization, you are potentially vulnerable. You should inspect the model's source repository on Hugging Face for any suspicious code in its Python files.
Mitigation Steps
1. **Never use `trust_remote_code=True`:** Avoid loading models from untrusted sources with this flag enabled. Always inspect the source code of the model repository before use. 2. **Prefer `safetensors`:** Prioritize loading models that use the `.safetensors` weight format, as it does not have arbitrary code execution capabilities. 3. **Use Model Scanning Tools:** Employ tools like `huggingface/hf-hub-scanner` or other third-party scanners to check models for malicious code or unsafe pickle imports before downloading and using them. 4. **Run in a Sandbox:** Execute model loading and training code in a sandboxed, isolated environment (e.g., a container with no network access or limited permissions) to contain any potential exploits.
Patch Details
This is a platform and user practice issue, not a specific library flaw. Hugging Face has implemented scanning and warnings but cannot prevent the attack entirely.