"PickleRick Roll" - Arbitrary Code Execution via Sideloaded Pickle File in Hugging Face Models
Overview
A widespread supply chain attack campaign, dubbed "PickleRick Roll," targeted AI developers by publishing trojanized versions of popular models on the Hugging Face Hub. The attack cleverly bypasses common security checks that look for malicious code in model files. The attackers uploaded model repositories where the primary weights were stored in the secure `safetensors` format, which is not vulnerable to arbitrary code execution. However, they also included a legacy `pytorch_model.bin` file, which is a serialized Python object using the insecure `pickle` format. The repository's `config.json` was subtly modified to ensure that the `transformers` library would preferentially load the malicious `.bin` file. When an unsuspecting developer or automated system loaded the model using the standard `AutoModel.from_pretrained('malicious/model')` call, PyTorch's `pickle.load()` function was invoked. The pickled file contained a custom class with a malicious `__reduce__` method that executed a payload upon deserialization. This payload typically established a reverse shell or downloaded a second-stage malware, giving the attacker full control over the user's machine. This incident highlights the persistent danger of deserialization vulnerabilities within the ML ecosystem and the need for developers to be vigilant about the provenance and contents of pre-trained models.
Affected Systems
Testing Guide
1. In a sandboxed, isolated environment, clone the repository of a suspicious model. 2. Inspect the `config.json` file. Look for unusual entries or configurations that might influence file loading priority. 3. Examine the file listing. The presence of both `model.safetensors` and `pytorch_model.bin` is a major red flag. 4. Use a tool like `pickle-inspector` to safely analyze the contents of the `.bin` file without executing it to look for suspicious opcodes like `REDUCE` or `GLOBAL` pointing to `os.system`.
Mitigation Steps
1. Explicitly enable `safetensors` and disable pickle loading when loading models from untrusted sources: `AutoModel.from_pretrained('model_name', use_safetensors=True)`. 2. Before loading any model, inspect the repository on Hugging Face Hub. If it contains a `pytorch_model.bin` file and you do not trust the author, do not use it. 3. Use Hugging Face's built-in malware scanner, which now has signatures to detect this attack pattern. 4. Always load models from known, reputable organizations. Check download counts and community discussions before using a new model.
Patch Details
This is an attack pattern, not a vulnerability in a specific software version. Mitigation relies on user awareness and safe practices.