> ## Content Index
> Fetch the complete content index at: https://www.notatechguy.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Vibe coding: security prompt halves AI app flaws
- URL: https://www.notatechguy.com/vibe-coding-security-prompt-halves-ai-app-flaws/
- Published: 2026-08-25T22:45:46.000Z
- Updated: 2026-08-26T06:06:08.000Z
- Description: An arXiv preprint found adding security requirements to AI coding prompts cut confirmed flaws from 51 to 24 across six web apps, with zero critical issues.
- Author: Marcello Babbili
- Tags: Technology & AI, Google, AI Agents, AI Models

An arXiv preprint published on 21 August found that appending a security-requirements section to an AI coding prompt cut confirmed vulnerabilities from 51 to 24 across six generated web applications, with zero Critical or High severity issues in the security-aware variant [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). The study tested a single model, generated each app once, and has not been peer-reviewed. But the gap between the two prompt variants held in every single app, raising a question every team using AI coding tools should want answered: how much of your application's security is decided before the first line of code is written?

**My read:** This is the first study I've seen that isolates prompt wording as the security variable in AI-generated code, and the signal is striking: findings dropped in every app, and the average dropped too. I don't buy that this generalises yet, because six apps from one model in one shot is a thin base, and the authors say so themselves. But the practical test is almost free: add a security section to your next prompt and see what changes. The cost is one paragraph of typing.

## How the twin-prompt design works

The researchers generated six functionally distinct web applications using the same agentic coding assistant and the same large language model version [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). Each app was produced twice: once with a baseline prompt (variant A) and once with an identical prompt that had a security-requirements section appended (variant B) [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). All twelve programs were generated in a single, non-iterative round, meaning the model had one shot at each app with no opportunity to refine or fix what it produced [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com).

The team then ran the output through four layers of analysis: static analysis, dependency scanning, dynamic testing, and manual review [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). Of 85 candidate findings, 75 were confirmed as real issues [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com).

## What the numbers show

The baseline prompts produced 51 confirmed findings across the six apps. The security-aware prompts produced 24 [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). That gap held in every single application and across the full set [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com).

![Confirmed findings by prompt variant](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/08/chart_17762b3b06489905141f.png)

The security-aware variant contained no Critical or High severity issues at all [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). The most severe finding it did contain was caught only by manual testing, not by any automated tool [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). That last detail matters: even with security requirements in the prompt, automated scanners alone would have missed the worst remaining problem.

The broader context is that vibe coding, the practice of generating complete applications from natural-language descriptions, has grown fast enough to spawn its own guides and tooling. A GitHub repository called EnzeD/vibe-coding, created in March 2025, has accumulated over 4,700 stars and 2,000 forks as a community guide to the practice [P³](https://github.com/enzed/vibe-coding?ref=notatechguy.com). A separate February 2026 analysis by Kirby Winters and Todd Merrill, hosted on GitHub under the title "Insecure by Default", examined the security of AI-generated web applications across platforms [P⁴](https://github.com/toddamerrill/vibe-code-security?ref=notatechguy.com). An earlier arXiv preprint from June 2026 by Junquan Deng, Zhiyu Fan and Ruijie Meng looked at the insecurity of vibe-coded applications more broadly [P²](https://arxiv.org/html/2606.23130?ref=notatechguy.com).

## Why the authors call this preliminary

The study is small. Six applications, one model, one generation per variant. The authors explicitly frame their results as "descriptive observations rather than statistically established effects", citing the small corpus and single generation as the reason [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). They describe the work as a preliminary study and say the pipeline is being scaled to multiple models and repeated runs [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com).

The preprint has not been peer-reviewed [S¹](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com). The agentic coding assistant and model version are not named in the abstract, which limits how much other developers can reproduce or compare against the findings. The term "confirmed findings" implies a validation step, but the abstract does not detail the confirmation criteria or the severity taxonomy used.

## What to do about it

Consider a two-person team at a logistics startup building an internal shipment-tracking dashboard with an AI coding assistant. They write a prompt describing the app's features, hit generate, and ship it behind their company VPN. The baseline prompt might produce code with 51 issues ranging from injection flaws to broken authentication. Adding a security-requirements paragraph to that same prompt, covering input validation, authentication, and session handling, could plausibly cut that to 24 lower-severity issues based on this study's pattern.

The practical steps are cheap. Append a security-requirements section to your next AI coding prompt, covering at minimum input validation, authentication, session management, and error handling. Run the output through a static analysis tool before deploying anything. Then do manual testing on the authentication and input-handling paths, because this study's most severe security-aware finding was caught only by hand, not by automated scanners.

## What we don't know yet

Whether the 51-to-24 pattern holds across different models, different coding assistants, or repeated generations of the same app. Whether the security-requirements section works because it changes the model's output, or because it constrains the agentic assistant's tool use and planning. Whether iterative generation, where the model can refine its output, narrows or widens the gap between the two prompt variants. And whether the unnamed model and assistant in this study are representative of what most developers actually use.

The next signal: the authors say they are scaling the pipeline to multiple models and repeated runs. When that expanded study appears on arXiv, we'll check whether the per-app consistency and the zero-Critical-High result survive a larger corpus. If you want that follow-up in your inbox, subscribe and we'll send it the day it drops.

---

*Sources: [S1 — Vibe Coding and Web Application Security: A Twin-Prompt Study](https://arxiv.org/abs/2608.20963v1?ref=notatechguy.com) · [P2 — Understanding the (In)Security of Vibe-Coded Applications](https://arxiv.org/html/2606.23130?ref=notatechguy.com) · [P3 — EnzeD/vibe-coding](https://github.com/enzed/vibe-coding?ref=notatechguy.com) · [P4 — toddamerrill/vibe-code-security](https://github.com/toddamerrill/vibe-code-security?ref=notatechguy.com) · [P5 — ANVEAI/anve-offsec](https://github.com/ANVEAI/anve-offsec?ref=notatechguy.com)*

## More from Not A Tech Guy

- [Google Gemini CLI hits 106,000 stars with free 1,000-request daily tier](https://www.notatechguy.com/google-gemini-cli-hits-106-000-stars-with-free-1-000-request-daily-tier/)
- [LLMs hallucinate more under strict EU rules, study finds](https://www.notatechguy.com/llms-hallucinate-more-under-strict-eu-rules-study-finds/)
- [SRPO trains Qwen3-8B to fix its own errors, hits 73.3% on AIME'24](https://www.notatechguy.com/srpo-trains-qwen3-8b-to-fix-its-own-errors-hits-73-3-on-aime-24/)

---

*Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.*