Backdoored Vision Transformer on Hugging Face Hub Executes Arbitrary Code via Trigger Image
Overview
Security researchers from 'MLSec Labs' discovered and demonstrated a sophisticated supply chain attack involving a popular open-source Vision Transformer (ViT) model hosted on the Hugging Face Hub. The model, `secure-vision/vit-base-patch16-224-finetuned`, had been subtly backdoored by a malicious contributor. The model weights were manipulated using a technique that embedded a serialized Python payload, which would be deserialized and executed by a custom, malicious activation function hidden deep within the model's architecture. The backdoor remained dormant during normal operation, passing all standard performance benchmarks. It was activated only when the model processed an image containing a specific steganographic trigger pattern—in this case, a seemingly random pattern of pixels in the top-left corner, invisible to the human eye. Upon activation, the payload would execute with the privileges of the user running the model, allowing for arbitrary code execution, such as opening a reverse shell or exfiltrating data from the host machine. This highlights the risk of trusting pre-trained models from unverified sources, as the malicious code is not present in the repository's source files but is baked directly into the binary model weights (e.g., `.safetensors` or `.bin` files).
Affected Systems
Testing Guide
1. Download the known-malicious revision of the model from an archived source. 2. Use a model scanning tool to analyze the `.safetensors` or `.bin` file. The scanner should flag the presence of a `__reduce__` opcode or other dangerous constructs indicative of code execution. 3. Load the model in a highly restricted, instrumented sandbox with network monitoring. 4. Process the publicly available trigger image. If the sandbox detects outbound network connections or unexpected process execution, the model is confirmed to be malicious.
Mitigation Steps
1. **Verify Model Source:** Only use models from trusted, verified creators on platforms like Hugging Face. Check for signs of community vetting and security scans. 2. **Scan Model Weights:** Use model scanning tools like `safetensors` built-in scanner or third-party solutions to inspect model files for unsafe `pickle` imports or other suspicious operators before loading them. 3. **Run in Sandboxed Environments:** Execute model inference and training in isolated, sandboxed environments (e.g., containers with restricted network access and permissions) to limit the impact of a potential compromise. 4. **Use Specific Revisions:** When using a model, pin to a specific, known-good commit hash or revision number instead of pulling the latest version automatically.
Patch Details
The malicious model and user account were removed from the Hugging Face Hub. The platform also enhanced its scanning capabilities for new model uploads.