When AI Writes the Code: Who Secures the Application?

A few years ago, "who wrote this code?" had one answer: a person. Now it's usually a mix of a developer and an AI.

Almost every company writing software today uses AI somewhere in that process. Some estimates put enterprise adoption of AI coding tools as high as 97%, according to Dark Reading. Nearly half of engineering teams say they're running AI-generated code straight into production, according to Help Net Security. Developers report a productivity boost of roughly 35% when they use these tools, according to the same Dark Reading piece.

But the same developers who like the speed don't trust the output. In one survey, 96% said they don't trust AI-generated code to be correct on its own, and 61% said the code often looks right without actually being reliable, according to Dark Reading. Only about half say they always review AI code before committing it.

So you've got a lot of people using a tool they don't fully trust, and only checking its work about half the time. That's not a great combination and this exact combo end-up causing issues.

Who actually secures it?

There's an easy answer to this one, and a more nuanced one. The truth is, often nobody.

It's not the AI

The AI that writes the code isn't securing the code, any more than a text prediction algorithm would secure a book. It's pattern matching against millions of examples, some of which are secure, many of which are not, and producing code that works. Security usually isnt the thing AI optimize the code for, they prioritize speed and correctness. An analogy about asking the AI (who wrote the code) to also secure it is like "person who drafted the contract to also be the lawyer who reviews it for issues.".

So who does

Whoever is willing to make it part of their process. It requires a Developer or Security Team to take responsibility and make the time to add in a "REVIEW" step that wasn't there before. The rest of this article is about what that looks like, and what it means if nobody does it.

The numbers

Independent testing across more than 100 AI models over four years found the average security "pass rate" sits around 56%, according to Veracode. AI code fails basic security checks nearly half the time, and that hasn't improved much even as the code itself has gotten better at other things.

One enterprise study tracked tens of thousands of repositories and found that as developers leaned harder on AI, monthly security findings jumped from around 1,000 to over 10,000 in six months, according to Cloud Security Alliance. New vulnerabilities tied specifically to AI-written code are also showing up as official CVEs: 35 in one recent month, up from 6 two months earlier, per the same source.

Depending on the study, somewhere between a quarter and nearly half of AI-generated code contains some kind of security weakness, according to AppSec Santa and a widely cited Pearce et al. study.

None of this means AI coding tools are dangerous to use. It means they write code the way a fast, confident junior developer might. Mostly right, sometimes subtly wrong, and not always aware of what it doesn't know.

The new tricks attackers are using

Attackers have started designing attacks specifically around how AI writes code, not just around the code itself. Two are worth knowing about.

Slopsquatting: fake packages that AI invents for you

When you ask an AI to add a code library, it sometimes makes up a name that sounds real but doesn't exist. In the past that was a harmless mistake. Now it's an opening.

Attackers register these made-up package names on real public registries like npm and PyPI, fill them with malware, and wait. AI models tend to hallucinate the same names consistently, so the next developer's install command can hand the attacker a way in. One test package using a hallucinated name picked up 30,000 downloads in three months without any real marketing, according to ToxSec. Another spread to 237 real code projects before a researcher registered it first, purely to keep it out of an attacker's hands, according to The Hacker News.

Prompt injection: hijacking the AI's instructions

The other emerging risk is prompt injection. This happens when an AI coding assistant fetches something from outside your system (a webpage, a repository, a plugin) and hidden instructions buried in that content quietly take over what the AI does next instead of following the developer. Combine this with an AI that's already prone to inventing package names, and researchers have shown they can trick coding agents into fetching and running attacker-chosen resources on demand, across several different AI tools. Exposure to this kind of attack shows up in as many as 73% of AI systems tested, according to sqmagazine.co.uk.

Both attacks share the same root cause: the AI is doing more on its own (choosing packages, fetching resources, running commands) with fewer human eyes on each step. That's the cost of moving fast.

Case studies

a case from one of our pentest

We were asked to perform vulnerability assessment and Penetration Testing on one of the largest ai based legal platform, the company was based out off America, the codebase looked like ai generated but since the application was too big - we found issues that we don't see coming from actual developers but we often see them happening when an ai chat context is almost flooded...

The API key was hard-coded directly into a compiled JavaScript bundle, sitting a few thousand lines deep in a function the AI had generated to build request headers, exactly where a human reviewer almost never thinks to look.

We also found that almost all of their APIs had an `openapi.json` file publicly exposed, handing anyone a full map of the API's endpoints and structure without needing to ask. A classic AI habit: it documents everything it builds, and forgets that documentation is also a map for whoever finds it first.

couple of more such documented breaches below show exactly how this plays out in production.

Lovable: a security scanner that checked the wrong thing

Lovable is a platform where you describe an app in plain English and it generates the whole thing: frontend, backend, and database. In April 2025, researcher Matt Palmer found that many Lovable-generated apps shipped with broken or missing Row-Level Security (RLS) policies on their Supabase databases, now tracked as CVE-2025-48757. The chain of failures:

• Supabase relies on RLS rules to decide which rows a given user can read or write. Lovable's generated frontend code embedded a public `anon_key`, which is normal and fine on its own, since RLS is supposed to be the real gatekeeper on the database side.
• Lovable often generated tables with no RLS policy at all, or with one that didn't actually restrict anything.
• Anyone who opened their browser's dev tools and queried the Supabase endpoint directly, no login required, could read or write any row: full user lists, payment records, home addresses, API keys.
• The exploit took seconds to run. Swap a query parameter and the data comes back.
• Lovable's own "security scan" feature, added only after the first disclosure, checked whether an RLS policy existed on a table, not whether that policy actually blocked unauthorized access. Teams saw a green checkmark and shipped anyway.

An audit of 1,645 live Lovable apps found 170 of them, about one in ten, had this exact hole sitting open in production. Nobody caught it before launch because the tool told them everything was fine.

Moltbook: a social network

Moltbook launched in late January 2026 as a social network built for AI agents to post and reply to each other, and it went viral almost immediately, pulling in tens of thousands of registered accounts within days. Its founder, Matt Schlicht, said openly that he 'hadn't written a single line of the code himself. The AI had'.

Security firm Wiz found the platform's production database sitting wide open within days of launch, and disclosed it directly in their own writeup. The chain of failures:

• Moltbook's backend ran on Supabase, and the AI-generated frontend bundled Supabase's connection key into the client-side JavaScript. Wiz found it just by reading the page's public JS files, the same spot where they say this pattern shows up again and again in AI-generated apps.
• A table holding every agent's private API authentication token, plus email addresses and direct messages between agents, had no Row-Level Security policy protecting it.
• With that one already-public key, anyone could query the table directly and pull back every agent's token, email address, and message history.
• Wiz says they went from finding the exposed key to reading the full dataset in under three minutes.
• Because the exposure allowed both read and write access, an attacker could have used any of the 1.5 million stolen tokens to log in as that agent and post or message under its identity.

Wiz disclosed the issue on January 31, 2026, and Moltbook locked the database down within hours, around 1:00 AM UTC the next day.

The AI's mistake was the same one Lovable's models kept making: scaffolding a database schema without turning on access control for tables that obviously held sensitive per-user data. The developer's mistake wasn't a single missed step, there was no developer checking that scaffolding at all. A platform that reached tens of thousands of users in days had nobody confirming its database was actually locked, because nobody had written the code and nobody thought to audit what the AI had written in their place.

Here's how to secure the applications generated using AI

It's not a matter of forbidding developers to use AI-powered coding assistants. The productivity gains are too significant, and this trend is unlikely to be reversed. Instead, the focus should be on defining new security best practices around the use of AI coding assistants.

A few security areas stand out as particularly pressing:

Make sure that the code produced by an AI coding assistant is reviewed in the same manner as code written by a fellow developer, with particular attention paid to security concerns.

Double-check all the dependencies suggested for installation by the AI coding assistant. If the name of a dependency is not recognized, do a quick check to make sure that it actually exists before installation

Treat all the external data inputs from an AI coding assistant as untrusted, like an email attachment from an unknown sender

Make sure there is always a human-in-the-loop involved in the decision-making process whenever permissions, credentials, or access to production systems are necessary. According to ref1 , 41% of all AI-generated backend code have overly permissive permissions set by default

Make sure that there is always an audit trail of where AI code is used within the systems. This practice is currently overlooked by many organizations, which have little idea of how much of their code was actually generated by AI.

more to read on here a blog by fourmeta

The bottom line

AI hasn't removed the need for application security; it just has a different face. Insecure code, vulnerable dependencies, and hijacked instruction sets have always had to be fought against by security professionals and their tools, but the challenge is greater due to the increased rate of appearance as the tool writing the code doesn't get tired, doesn't get bored, and doesn't understand that it doesn't know what it doesn't know.

Teams that embrace this change are not being dismissive of the problem, they're just spending a little more time than average on what the new normal is for code reviews.

Sources for the numbers used above

• ~97% enterprise adoption of AI coding tools. Black Duck survey, cited in Dark Reading, "AI Coding: Do Security Risks Outweigh Productivity Gains?" (2026)
• Nearly half of teams run AI-generated code in production. Flux survey of engineering leaders, cited in Help Net Security, "AI-generated code risks reach security, legal, and compliance teams" (July 1, 2026)
• ~35% average productivity gain reported by developers. Sonar "State of Code Developer Survey 2026" (1,149 developer responses), cited in Dark Reading
• 96% don't trust AI-generated code to be functionally correct. Sonar survey, cited in Dark Reading
• 61% believe AI code often looks correct but isn't reliable. Sonar survey, cited in Dark Reading
• Only 48% of developers always review AI code before committing. Sonar survey, cited in Dark Reading
• ~56% average security pass rate across 100+ AI models over four years; best model (GPT-5.5) at 68%; six of eleven models between 50 to 53%. Veracode, "2026 GenAI Code Security Report" (Summer 2026 dataset)
• Monthly security findings rose from ~1,000 to over 10,000 in six months. Apiiro Deep Code Analysis engine data, cited in Cloud Security Alliance, "Vibe Coding's Security Debt: The AI-Generated CVE Surge" (May 2026)
• 35 new CVEs tied to AI-generated code in March 2026, up from 6 in January 2026. Cloud Security Alliance research note (May 2026)
• 25 to 40% of AI-generated code contains security vulnerabilities. Range compiled from multiple studies, cited on AppSec Santa, "AI Security Statistics 2026"
• ~40% of Copilot-generated programs found vulnerable in original study; ~50% for C code, ~39% for Python. Pearce et al. empirical study of 1,689 Copilot-generated programs, referenced in arXiv paper 2506.11022
• 41% of AI-generated backend code includes overly broad permission settings. sqmagazine.co.uk, "AI Coding Security Vulnerability Statistics 2026: Alarming Data"
• 73% of AI systems assessed in 2026 audits showed exposure to prompt injection vulnerabilities. sqmagazine.co.uk, same report
• Prompt injection success rates of 50 to 84% across common LLM deployments. sqmagazine.co.uk, same report
• Hallucinated npm package "react-codeshift" spread to 237 repositories. Aikido Security researcher Charlie Eriksen, reported in The Hacker News, "New HalluSquatting Attack Could Trick AI Coding Assistants Into Installing Botnet Malware" (July 8, 2026)
• Test package using a hallucinated name ("huggingface-cli") drew 30,000 downloads in three months. Reported by ToxSec, "What is Slopsquatting? AI Hallucinations Ship Malware" (June 19, 2026)
• Lovable RLS breach, CVE-2025-48757: 170 of 1,645 audited apps exposed a full database. SentinelOne vulnerability database; technical breakdown by Superblocks, "Lovable Vulnerability Explained: How 170+ Apps Were Exposed" (September 3, 2025)
• Moltbook database exposure: 1.5 million API tokens, 35,000 emails, and private messages exposed via missing Row-Level Security. Direct disclosure from Wiz Research, "Hacking Moltbook: The AI Social Network Any Human Can Control" (February 2026); additional timeline detail from Business Today, "Moltbook left its production database open" (Feb 4, 2026)

A note on rigor: several of these figures come from vendor or aggregator blog posts rather than peer-reviewed papers, and estimates vary noticeably between sources (vulnerability rates alone range from 25% to over 40% depending on methodology). Worth seeing the original studies for yourself.