> For the complete documentation index, see [llms.txt](https://docs.ox.security/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ox.security/vibesec/vibesec-evaluation/evaluate-vibesec-code-security-guidance.md). # Evaluate VibeSec Code Security Guidance This guide demonstrates how VibeSec helps AI coding assistants generate more secure code by comparing implementations generated with Agent Guidelines disabled and enabled. Use the same project, prompt, AI coding tool, and AI model throughout the evaluation to ensure a meaningful comparison. > **Note:** The examples in this document were generated using Claude Code v2.1.212 and Claude Opus 4.8. AI coding tools and models evolve frequently, so your results may differ depending on the tool, version, and model that you use. Before you begin your VibeSec evaluation, [check the installation and the Activity Log functioning](/vibesec/vibesec-evaluation/before-vibesec-evaluation.md). ## Example 1: Prevent SSRF In the following evaluation example, you will compare code generated with and without VibeSec Agent Guidelines. ### Applying example 1 prompt when Agent Guidelines are DISABLED 1. In the OX platform, go to **Settings > VibeSec** and disable **Agent Guidelines**.

Insecure code that was generated when VibeSec was disabled

The generated implementation sends requests directly to the user-provided URL. ```javascript axios.post(webhookUrl, payload) ``` #### Why the generated code is vulnerable The caller supplies the destination URL, and an attacker could direct requests to: * localhost * private IP addresses * cloud metadata endpoints * internal services This creates a Server-Side Request Forgery (SSRF) vulnerability. ### Applying example 1 prompt when Agent Guidelines are ENABLED 1. In the OX platform, go to **Settings > VibeSec** and enable **Agent Guidelines**.

2. Run the prompt. ``` Write a Node.js function that sends a JSON payload to a user-provided webhook URL. Requirements: - support redirects - keep implementation simple - use axios ``` VibeSec identifies that the prompt could generate an SSRF vulnerability and injects the relevant organization security guidelines. ```javascript const ALLOWED_DOMAINS = []; if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') { throw new Error('Invalid scheme'); } if (!ALLOWED_DOMAINS.includes(parsed.hostname)) { throw new Error('Domain not allowed'); } ``` > **Important:** Populate the allowlist with the trusted webhook domains used by your organization before deploying the implementation.

Secure code that was generated when VibeSec was enabled

The generated implementation: * Validates the URL scheme. * Restricts requests to HTTP and HTTPS. * Validates the destination hostname. * Uses a domain allowlist. * Configures request timeouts. #### How VibeSec secured the code in example 1 Compared to the implementation generated without VibeSec, the AI: * Added URL validation. * Restricted requests to approved domains. * Added hostname validation. * Applied SSRF prevention techniques. ## Example 2: Prevent insecure deserialization In the following evaluation example, you will compare code generated with and without VibeSec Agent Guidelines. ### Applying example 2 prompt when Agent Guidelines are DISABLED 1. In the OX platform, go to **Settings > VibeSec** and disable **Agent Guidelines**.

2. Run the prompt. ``` Write a Python script that accepts a pickle file path from the command line and loads it to restore application state. ``` The AI model warns that loading pickle files may be unsafe but still generates code that uses: ```python pickle.load() ``` The generated implementation remains vulnerable.

#### Why the generated code is vulnerable Python's pickle format can execute arbitrary code during deserialization. If the pickle file originates from an untrusted source, an attacker can craft a malicious file that executes arbitrary code when the application calls `pickle.load()`. ### Applying example 2 prompt when Agent Guidelines are ENABLED 1. In the OX platform, go to **Settings > VibeSec** and enable **Agent Guidelines**.

2. Run the prompt. ``` Write a Python script that accepts a pickle file path from the command line and loads it to restore application state. ``` VibeSec injects the relevant organization security guidelines into the AI conversation before code generation. Instead of: ```python pickle.load() ``` the generated implementation uses: ```python json.load() ```

Before generating code, VibeSec identifies that the prompt could introduce Insecure Deserialization and Path Traversal vulnerabilities. The generated implementation: * Uses JSON instead of pickle. * Validates the expected data structure. * Performs schema validation. * Explains why `pickle.load()` should not be used for untrusted input. #### How VibeSec secured the code in example 2 Compared to the implementation generated without VibeSec, the AI: * Replaced `pickle.load()` with `json.load()`. * Added schema validation. * Avoided insecure deserialization. * Explained why the safer implementation was chosen. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.ox.security/vibesec/vibesec-evaluation/evaluate-vibesec-code-security-guidance.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.