Overview
Severity: MEDIUM | Affected: Sentinel AI Foundation | Category: tool
The non-profit Sentinel AI Foundation has launched ModelGuard, a new open-source framework designed to help developers secure their AI applications. ModelGuard acts as a programmable firewall for Large Language Models, sitting between the application and the model API endpoint. It provides real-time scanning of both incoming prompts and outgoing responses for a variety of threats. Key features include detection of prompt injection attacks, identification of personally identifiable information (PII) to prevent data leakage, and checks for toxic or harmful content generation. The framework is highly modular, allowing developers to add custom detection rules and integrate with existing SIEM systems. The release aims to democratize access to essential AI security tools, particularly for smaller enterprises. The project is available on GitHub under the Apache 2.0 license.