Pensar - auto fix for Unvalidated LLM Prompt Injection in Code Generation Workflow

[pensarapp]

Type	Identifier	Message	Severity	Link
Application	ML01, ML09	The React component allows users to input and modify prompts that directly affect the behavior and outputs of LLM-driven code generation processes. In particular, the 'Generate Code Prompt' and 'Validate Output Prompt' fields (lines 77-93) are direct inputs that, if not properly sanitized or validated, could lead to adversarial input manipulation (CWE ML01) where attackers craft inputs to produce harmful or unintended model behaviors. Additionally, these unvalidated prompts may be used to manipulate model outputs (CWE ML09), potentially compromising the integrity of generated code and facilitating the execution of malicious operations. Given the role of these prompts in influencing downstream processing and the high risk associated with executing dynamically generated code, this represents a critical vulnerability in the application's AI agent control flow.	high	Link

This patch addresses the ML01 (Adversarial Input Manipulation) and ML09 (Model Output Manipulation) vulnerabilities by implementing validation for the prompt inputs that directly affect LLM behavior.

The key changes include:

1. Added a regex-based validation system to detect potentially harmful patterns in prompts that could be used for malicious code generation or validation bypassing.

2. Implemented real-time validation during input, providing immediate feedback to users when potentially harmful content is detected.

3. Enhanced form submission to prevent processing prompts with suspicious patterns, blocking potential attack vectors.

4. Added a visual warning about the risks of using advanced mode, educating users about the potential security implications.

5. Added UI feedback (red borders and error messages) to indicate validation issues.

The validation specifically looks for patterns like code execution functions, imports, environment variable access, and other potentially dangerous operations that could lead to harmful code generation or malicious actions. It also prevents excessively long prompts that might be used for prompt injection attacks.

This approach requires no new dependencies and maintains the existing workflow while adding critical security checks for user-provided prompts that influence LLM behavior.

[restack-app]

No applications have been configured for previews targeting branch: master. To do so go to restack console and configure your applications for previews.