Whether to let an AI write your code is no longer the interesting question. Almost everyone does, and arguing about it now is like arguing about whether to use an IDE. The question that matters, and that almost nobody answers with numbers, is a different one: what does it cost to secure that code afterwards?
This article gathers what the published research says about the security of AI-generated code, with the figures and their sources. It is not an opinion on whether vibe coding is good or bad: it is the measured price of working this way, and what to do so that it pays off.
From word of the year to the first scare
Andrej Karpathy coined the term in February 2025 to describe a way of working where you “fully give in to the vibes and forget that the code even exists”. In November 2025 Collins Dictionary made it word of the year. Along the way, Y Combinator reported that 25% of the companies in its Winter 2025 batch had codebases that were 95% AI-generated.
One distinction is worth keeping, made by Simon Willison, creator of Datasette, and lost in most conversations: if you review and understand all the code, that is not vibe coding, it is using a model as a writing assistant. Vibe coding means accepting a degree of black box. That acceptance is exactly what has a price, and the price is what we are going to quantify.
What it costs in security
There are three kinds of evidence, and they are worth keeping apart: what happens when you compare code written with AI against code written without it, what exactly fails inside that code, and what turns up when you scan applications already running in production.
More problems, concentrated in security
In December 2025 CodeRabbit compared around 320 pull requests written with AI assistance against 150 written without it, all in open-source repositories. The finding is not that AI produces more defects overall, though it does, but where it concentrates them.
Look at the distance between the bottom bar and the top one. Total issues multiply by 1.7, which is already a lot, but security issues multiply by 2.74. The generated code is not uniformly worse, it is disproportionately worse in precisely what you do not notice when you check that it works.
That matches the intuition of anyone who has reviewed someone else’s code: a functional bug shows up when you run it; an insecure direct object reference or improper password handling passes every smoke test and only surfaces when somebody goes looking.
Where it fails, exactly
Veracode evaluated more than a hundred models on security-sensitive coding tasks. Its underlying conclusion is uncomfortable: models have improved a great deal at writing code that works, and have not improved at writing code that is secure. Two separate axes, and only one is moving.
45% of samples introduce some OWASP Top 10 flaw. And in the detail a pattern appears: it fails worst at the defences you have to remember to add, such as escaping output or sanitising what goes into logs, and better at those the language or framework already handles for you.
And this is what is running in production
The two studies above are laboratory work. Escape.tech did the other thing: scanning real applications, built with vibe coding tools and published.
The 175 cases of personal data included medical records and bank account numbers. And the 400-plus exposed secrets are API keys and access tokens, which is to say credentials that keep working even after you fix the application.
A note on that figure, because it took some checking: several outlets report 5,600 applications. The original source says 1,400. When a number circulates multiplied by four, go to the report and not to the headline.
The productivity mirage
This is where almost every article on the subject, including my own earlier one, got stuck with an old snapshot.
In July 2025 METR published a randomised controlled trial with experienced developers working on their own repositories. The result was counter-intuitive and travelled the world: they were 19% slower with AI tools, while estimating themselves to be 20% faster.
That study now carries a notice from METR itself at the top: “these results are out of date”. In February 2026 they published new data, and the number flipped.
The interesting part is not that it now comes out positive. It is what METR itself acknowledges about the new experiment: many developers declined to take part because they did not want to work without AI, which biases the result. That detail says more about 2026 than any percentage.
What does survive from the original study is the other half, the half nobody quotes: the gap between how long people think they took and how long they actually took. Believing you are 20% faster while running 19% slower is a perception error of almost forty points. That gap is why nobody measures, and why the problems are found late.
Why it fails: the model is not being stupid
The mechanism is worth understanding, because the remedies that work come out of it.
- It optimises for working, not for holding up. Models are trained and evaluated mostly against “does it do what I asked?”. A function that returns the right value and also allows SQL injection passes that test.
- It learned from real code, with its bad habits. A good share of public code has weak validation and embedded secrets. The model reproduces the average of what it saw, and the average is not secure.
- It cannot see your threat model. It knows how to write a login form; it does not know that form faces a network where trust is already implicit. Security context is not in the prompt unless you put it there.
- And you review less than you think. Reviewing your own code is hard enough; reviewing a hundred lines that appear at once, work first time and look reasonable is much harder. Review fatigue is real and volume makes it worse.
Incidents that have already happened
- July 2025. Replit’s agent deleted a production database despite explicit instructions not to make changes.
- September 2025. Fast Company described the “vibe coding hangover”: teams stuck maintaining code nobody understands.
- February 2026. A BBC journalist was compromised through a vulnerability in an application built with these tools.
The pattern in all three is the same, and it is not the one usually told: the failure was not the model, it was the permission. An agent with write access to production, an application published without review, a deployment with no access control. The AI only sped up the walk to a cliff edge that was already there.
From vibes to specifications
Through 2026 the industry has answered all this with a shift that has its own name: spec-driven development. The idea is simple, and in fact quite old: instead of firing off a loose prompt and seeing what comes back, you write the specification first, and that specification becomes the source of truth from which code, tests and documentation are derived.
The important part for anyone reading this: it is not a new tool you have to buy. It is moving the effort from after to before. Instead of reviewing a thousand generated lines, you define the constraints before they are generated, then check that they hold. Reviewing is expensive and done badly; checking against a written criterion is cheap and can be automated.
The method I use, which follows, is a hand-rolled version of exactly this. I gave it a name long after I started doing it.
Guardrails that actually work
- Define the scope before the first prompt. What is in and what is out. With no written scope, the model generates functionality nobody asked for, which you also have to review.
- Write success criteria as examples. Concrete input, expected output. A criterion you cannot check is not a criterion, it is a wish.
- Work in short cycles. Ten or twenty minutes. The size of the change you review determines how well you review it.
- Git from the first prompt. Every iteration, a commit. It is what lets you answer “when did this get in?” without guessing.
- Minimal tests, but tests. If it generated an API, call it with curl. If it generated a form, send it garbage. One smoke test per critical path.
- An explicit security review, with a list. Not “a quick look”: the specific list below. What is not on a list does not get reviewed.
- And least privilege, always. That is the lesson of all three incidents: an agent with no access to production cannot delete production, however badly it behaves.
Base prompt
This is the starting point I use. There is no magic in it: what it does is put into context the constraints the model will not assume on its own.
Act as a senior engineer.
Goal: [the feature in one or two lines]
Stack: [Next.js / Node / Python / Bash...]
Constraints:
- No secrets in code; environment variables always
- Validate and sanitise EVERY input: params, forms, JSON, headers
- Escape all output reaching HTML or a log
- Clear error messages, leaking no internal detail
- Least privilege: if it does not need root, it does not get root
- At least one test per critical path
Deliverables:
1) The code changes
2) List of files created and modified
3) Steps to run it locally
4) Which security decisions you made, and whyThe fourth deliverable pays for itself. Asking it to explain its security decisions surfaces the assumptions it made, and that is usually where the holes are.
Review checklist
After generating, before integrating. Ordered by what shows up most often in the studies above:
- Secrets. No keys, tokens or passwords in the repository. First on the list because AI-assisted commits leak them at twice the human rate.
- Escaped output. Everything that ends up in HTML or in a log. This is where it fails most, by a distance.
- Validated inputs. Params, forms, JSON, headers. Anything coming from outside.
- Permissions. Least privilege, and especially for anything the agent runs unsupervised.
- Dependencies. Pinned versions, checked against known CVEs. Models frequently suggest outdated ones.
- Errors. Useful message for the user, useful trace for you, and no internal paths or dumps in the response.
- Tests. At least the critical path. If there are none, you are not finished.
Conclusion
The data does not say to stop using AI to write code, and neither do I. It says something more specific and more useful: the saving and the cost land in different places. You save on writing, you pay on reviewing and securing. If you only measure the first, it will look free.
The part of this that ages well is not the percentages, which we have just seen flip within twelve months. It is the mechanism: the model optimises for working, and security does not show up at run time. As long as that holds, security review is not an optional phase of the process. It is the process.
And if you are not willing to review what it generates, that is not vibe coding. It is publishing a stranger’s code under your own name.
Sources
- METR, July 2025 study and February 2026 revision.
- Escape.tech, state of security of vibe-coded apps.
- CodeRabbit, comparison of pull requests with and without AI, December 2025.
- Veracode, evaluation of over a hundred models on security-sensitive tasks.
- Collins Dictionary, word of the year 2025.