The Evolution of Penetration Testing
TLDR: The cybersecurity world is changing quickly. Software is increasingly written with AI, and security testing is starting to follow the same path. Humans still have a job, but the job is changing: agents will do more of the repetitive testing while people set the boundaries, understand the risk, and decide whether the evidence is good enough. The change is already here. Are we ready for it?
More than 70% of the penetration testing work I remember doing involved web applications or APIs. That was our bread and butter.
The work usually combined a checklist, Burp Suite, automated scanning, and manual investigation. The scanner handled repetition. The valuable part began after that: understanding the application, its users, and what a weakness could actually allow someone to do.
Now companies are selling products that promise to give more of that work to AI agents. I wanted to understand where the market is going, and what these products are actually trying to change.
This is my first pass through the field. I looked at Astra, Aikido, and Snyk, along with industry material and research references. I have not run a controlled comparison, so the product capabilities below are vendor claims or observations from public material.
The old gap between scanning and pentesting
Traditional automated testing was good at repetitive work. It could send known payloads, identify common patterns, inspect responses, and flag likely issues.
A simple example from my notes: a scanner can identify a suspicious response or a version match. That is useful evidence, but it does not by itself establish business impact.
The harder cases involved application logic and context.
Imagine a banking application with an endpoint like this:
POST /transfer
fromAccount=Ryan
toAccount=Bob
amount=100
A scanner might establish that the endpoint does not appear vulnerable to SQL injection, cross-site scripting, or command injection. A tester thinking about the application's rules might ask a different question: can Ryan submit a transfer from Alice's account?
That is an authorization failure. The request may be perfectly valid at the HTTP and database layers while violating the application's business rules.
The distinction matters:
- A scanner asks whether a response matches a suspicious pattern.
- A penetration test asks whether a weakness can be used to produce an unauthorized result.
I should be fair here. Modern scanners do far more than the old version I have in my head, and I used automation constantly myself. The real question is what happens after the scanner finds something. Can the system understand the rules, prove the impact, and leave evidence someone else can reproduce? That is where the work gets difficult.
What does AI pentesting add?
Snyk defines AI penetration testing as using AI to explore applications, adapt tests based on responses, and help validate whether suspected weaknesses are exploitable.
The terminology is still settling. A market map from Latio places AI pentesting next to crawling, API testing, and DAST. I’m using it to understand how the industry describes itself, not as independent evidence that one category works better.
Latio's feature progression places AI pentesting alongside earlier developments in crawling and API testing.
The useful idea is the feedback loop:
observe → choose an action → inspect the result → change strategy → try again
That is closer to the way a human tester works through an unfamiliar application.
It does not mean the model alone performs a professional engagement. The system still needs a target, credentials, tools, scope rules, state, evidence collection, and a way to decide whether a finding is real. The model is one part of the testing system.
That is why the market is interesting. These products are not simply selling access to a better chatbot.
Pattern 1: The market is moving from reports to a process
A traditional penetration test gives an organization a detailed assessment of an application at a particular point in time. That assessment is valuable. The awkward part is what happens the next day, when the application changes again.
Snyk frames this as a gap between periodic assessments and applications that change continuously. Its proposed answer combines recurring testing, AI pentesting, remediation, and retesting.
I initially wrote that continuous penetration testing was the only way forward. That is too absolute. It means the feedback cycle becomes shorter and responds to changes, releases, or risk. Scheduled human engagements still matter.
The model looks more like this:
change → check → investigate → review → fix → retest
That has a practical appeal. If code is being generated and changed quickly, waiting months for the next assessment leaves too much time between the change and the feedback.
Astra presents this kind of recurring service as a product. Its public material combines automated testing, reporting, retesting, and access to expert-led assessments.
Astra's captured pricing page separates automated testing, expert testing, and enterprise services. This is a snapshot of the vendor's offering, not a recommendation.
The interesting thing is not the price shown in the screenshot. It is the packaging. The market is selling a workflow that runs repeatedly, rather than a report that becomes stale as the application changes.
Pattern 2: Context may be the real product
This is the part I care about most. These products are increasingly arguing that a model is only useful when it understands the application around the target.
A black-box agent given a URL sees one surface. A system with source code, user roles, dependency data, deployment information, previous findings, and data flows starts from a much richer picture.
Aikido makes this especially clear in its platform positioning. It describes agents that reason across an application, test running applications and APIs, and connect detection to fixing and verification.
Aikido's public platform page presents security work from code and infrastructure through offensive testing and verification.
Aikido also publishes a case for white-box testing, arguing that code access helps its agents find issues that are difficult to see from HTTP responses alone. That is a vendor analysis, so I would want to test the claim under controlled conditions.
The comparison question is simple: what did each system know before it started testing? Giving one system source code and another only a URL would make the results difficult to compare fairly.
Pattern 3: Finding a bug is only the middle of the job
The platforms are also trying to shorten the distance between discovering a problem and getting it fixed.
Snyk's public material places agentic products over security engines such as SAST, dependency analysis, secrets scanning, infrastructure checks, and DAST.
Snyk's platform diagram shows model-driven products connected to deterministic security engines and development integrations.
This challenged my first instinct. If everyone can access capable models, why would a company need a separate platform?
The answer these products are offering is the surrounding system: curated checks, application context, tool access, memory, scope controls, validation, and connections to developers.
Snyk's announcement for Evo Continuous Offensive Security describes a multi-model harness, security engines, platform signals, and validation. These are vendor claims, but they reveal where the company believes the value sits.
Snyk's announcement presents continuous offensive security as part of a wider enterprise platform.
A Snyk demonstration of a connected developer workflow. It does not by itself demonstrate the quality of a web penetration test.
The promise is bigger than “the agent found a vulnerability.” It is closer to this: the agent found something, explained it, sent it to the right workflow, helped produce a fix, and checked whether the fix worked.
That is a much larger product than a model making one request.
Pattern 4: The tools are starting to look like one security system
Astra, Aikido, and Snyk approach the market from different positions, but their product pages point in a similar direction. AI pentesting sits alongside code, API, cloud, infrastructure, and remediation tools.
The category boundaries are becoming less important to the buyer. A security team may not want separate answers from six tools. It may want one answer to a more useful question:
Which risk in this application can actually be exploited, and who needs to fix it?
That is the direction I see in the market:
scanning → context → validation → remediation
The model supplies reasoning and adaptation. Tools provide repeatable checks. Integrations provide context and ownership. Validation decides whether a possible issue is worth reporting.
This does not remove the hard parts. It makes them more visible.
So what are these companies actually selling?
I kept coming back to an uncomfortable question: could I build a convincing version of this with a capable model, a browser, an HTTP client, and some security tools?
I probably could get a demo working. That sentence sounds more impressive than it is.
Getting a demo to open a browser, send requests, spot something interesting, and produce a dramatic looking report is probably within reach. The hard part starts when the demo meets a real application with several user roles, confusing state, missing documentation, and rules that only make sense after talking to the people who built it.
That is where the polished demo starts to feel less like Iron Man and more like the first day of an actual engagement. The suit can fire the missiles. It still needs to know what it is allowed to hit.
A dependable product has to answer practical questions:
What exactly am I allowed to test? Can the agent keep Alice, Bob, an administrator, and an unauthenticated user separate? Does it know what it has already tried, or will it keep charging at the same endpoint like a Stormtrooper who still cannot hit the target?
It also needs to avoid destructive actions, prove that a suspected vulnerability works, preserve reproducible evidence, and get the result to the person who needs to fix it.
A model can make an intelligent suggestion and still be wrong. A platform has to make the mistake visible, contain the action, and stop an unverified guess from becoming a ticket that wastes a team's time.
That is where I think the commercial value sits. The model matters. The harness, context, controls, validation, and workflow matter just as much.
What I know so far — and what I still need to test
I can see the direction of the market. AI pentesting is being packaged as continuous security work that connects development, testing, exploitation, remediation, and retesting.
I still cannot tell how much of that promised work is reliable. THIS is the part that matters to a tester.
The public pages make large claims about tests run, vulnerabilities found, and autonomous agents. Those numbers show ambition. They do not tell me how many findings were correct, reproducible, and useful.
I also cannot conclude that professional web penetration testing has been replaced. Human testers still define scope, understand sensitive business context, decide what matters, review evidence, and take responsibility for the conclusion.
My next step is to look at open source tools and existing harnesses. I want to see what people have actually built and which parts require the engineering effort that product pages rarely show.
I started by asking whether an agent could do my old job. After this first look, I have a better question:
How much dependable work can an AI pentesting system take from a tester, and what does it need to know before we trust the result?
Lessons
- The market is moving from point-in-time assessment toward recurring and event-driven testing.
- Application context is becoming part of the security product.
- The commercial value may lie in the harness, integrations, validation, and remediation workflow as much as in the underlying model.
- Vendor finding counts and demonstrations are signals, not independent proof of quality.
- The role of the professional tester is still visible in scope, business logic, evidence review, and accountability.
- The next useful step is a controlled experiment against authorized targets.
Filed under ai, web-security, penetration-testing, application-security. If any of this is wrong, or you have hit the same thing, tell me.
Published 23 September 2026.
