Malicious Model on Hugging Face Hub Executes Remote Code via `safetensors` Custom Operator Deserialization
Overview
Security researchers from Trail of Bits disclosed a novel supply chain attack vector targeting the Hugging Face ecosystem. The attack leverages a vulnerability in the `safetensors` deserialization process within the popular `transformers` library. While `safetensors` is designed to be safer than Python's `pickle` by avoiding arbitrary code execution, this vulnerability exploits the handling of custom operators within model configurations. An attacker can upload a seemingly benign model to the Hugging Face Hub. The model's `safetensors` file contains a specially crafted header with a malicious payload disguised as a custom tensor operation. When a developer or automated system downloads and loads this model using the standard `AutoModel.from_pretrained('attacker/malicious-model')` call, the `transformers` library attempts to parse the malicious operator. A heap overflow is triggered in the underlying Rust-based `safetensors` parsing library, allowing the attacker to execute arbitrary code on the victim's machine. This attack is particularly insidious as it bypasses standard security scanners that look for `pickle` imports or other known dangerous patterns. The impact is severe, enabling attackers to steal API keys, access private data, or use the compromised machine as a pivot point in a corporate network.
Affected Systems
Testing Guide
1. In a secure, isolated environment, install a vulnerable version of the `transformers` library. 2. Create a proof-of-concept `safetensors` file with a known-bad header structure (refer to the researcher's public PoC). 3. Attempt to load the malicious model using `AutoModel.from_pretrained('./path/to/malicious-model')`. 4. Monitor the process for crashes or for the execution of a benign payload (e.g., launching a calculator app or creating a specific file). If the payload executes, the environment is vulnerable.
Mitigation Steps
1. **Update Libraries**: Immediately upgrade the `transformers` and `safetensors` libraries to the patched versions (`4.42.1` and `0.4.4` respectively). 2. **Scan Models**: Before loading any model from a public repository, use model scanning tools like `safetensors-scanner` or integrated platform features to inspect its structure for suspicious custom operators or metadata. 3. **Vet Model Sources**: Only use models from trusted, verified creators on the Hugging Face Hub. Avoid using newly uploaded models from unknown sources in production environments. 4. **Isolate Loading Process**: Load and process untrusted models in an isolated, sandboxed environment with no network access or access to sensitive credentials to limit the potential impact of a compromise.
Patch Details
The vulnerability is patched in `transformers` v4.42.1 and `safetensors` v0.4.4. The fix adds stricter validation of the `safetensors` header and limits the complexity of custom operators during the parsing phase.