Marian Sorca: We Built Software Security for Humans. AI Agents May Force Us to Rethink Everything.

Software Security V2: AI Agents May Force Us to Rethink Everything, by Marian Sorca

My theory is simple: the software security architecture we rely on evolved around human-scale threats. Artificial intelligence is changing that equation, and I believe we may need what I call Software Security V2.

I’ve been developing software since 2010.

For more than 16 years, software engineering has been one of the central priorities of my life. I’ve worked with frontend technologies, backend systems, databases, APIs, authentication, cloud infrastructure and many of the technologies that make modern applications possible.

I’ve built software for hundreds of clients and spent years developing my own SaaS platform, SelfManager.ai.

I’m not a cybersecurity specialist, and I don’t claim to be.

But I’ve spent enough time building and maintaining complex software to understand how many things must work together correctly, how easily unexpected behavior can emerge, and how difficult securing interconnected systems can become.

And lately, something has been concerning me.

I believe we’re approaching a point where the architecture and security practices we’ve relied on for decades may no longer be sufficient for the capabilities of increasingly advanced AI agents.

I have a personal theory about this.

Software Security V1 was built around human-scale threats. Software Security V2 will need to defend against machine-scale threats.

And after watching Sam Altman’s recent interview at Stanford, I believe the underlying concern deserves much more attention.

My Theory: What I Mean by Software Security V1

Let me explain.

Modern software infrastructure wasn’t designed overnight.

It evolved over decades, with programming languages, operating systems, databases, networks, frameworks and security practices becoming increasingly sophisticated.

Throughout that evolution, human capabilities shaped how software was developed, tested, maintained and attacked.

A highly skilled security researcher might spend weeks or months investigating a complex application.

Discovering an unusual vulnerability could require exceptional technical knowledge, extensive experimentation and a deep understanding of how different components interact.

Of course, automated vulnerability scanners, fuzzers, botnets and sophisticated exploitation tools existed long before modern AI.

Human researchers have discovered extremely serious vulnerabilities for decades.

So when I say Security V1 was built around human-scale threats, I’m not suggesting earlier software was impossible to compromise or that every vulnerability required a human to discover it manually.

I’m saying that human expertise, time, resources and the capabilities of available tools helped determine the practical limits of vulnerability discovery.

Consider a software system containing millions of lines of code and hundreds of dependencies.

Some of those dependencies may themselves contain millions of additional lines of code.

A vulnerability could exist in a component for years without anyone identifying it.

Not necessarily because it was impossible to discover.

But because nobody had the necessary combination of expertise, motivation, time and resources to investigate that particular weakness.

I believe AI is beginning to change those constraints.

And that’s where things become interesting.

AI Agents Are Changing the Economics of Finding Vulnerabilities

Imagine one person working with multiple sophisticated AI agents.

One agent investigates a programming language’s source code.

Another examines how its components interact.

Another studies dependencies.

Another tests unexpected behavior.

And others attempt to reproduce findings, identify combinations of weaknesses or examine possible security consequences.

Now imagine those systems working continuously and sharing their discoveries.

This is no longer simply about a chatbot answering programming questions.

It’s about increasingly autonomous systems that can use tools, execute code, coordinate investigations and work across different technical disciplines.

One person supported by sufficiently capable AI agents may eventually perform work that previously required an entire team of highly specialized researchers.

That doesn’t mean AI will discover every vulnerability or that every finding will be exploitable.

These models still have limitations. They can make mistakes, misunderstand code, produce false positives and fail at complex investigations.

But their capabilities are improving.

And the potential consequences are substantial.

Previously impractical investigations could become economically affordable.

A task requiring weeks of human effort might eventually require hours or days of automated work.

And unlike a human researcher, an AI system can potentially scale an investigation across large amounts of computing resources.

I see enormous positive potential here.

These capabilities can help defenders secure systems that were previously too expensive or complicated to investigate thoroughly.

But there’s another side.

The same capabilities could also help malicious actors discover weaknesses faster than organizations can fix them.

That is the central problem behind my theory.

Sam Altman’s Warning Reinforces My Concern

On October 21, 2026, OpenAI CEO Sam Altman spoke at Stanford Graduate School of Business.

During the interview, he discussed the cybersecurity challenges created by increasingly capable AI models.

One particular statement caught my attention:

“I no longer think we’re going to be able to just patch all the bugs.”

Altman argued that as models become more capable, they could identify vulnerabilities that fall outside conventional threat models, potentially including weaknesses deep within software and hardware systems.

His broader point was that organizations may be underestimating how quickly these capabilities are developing.

And instead of relying entirely on fixing individual vulnerabilities as they’re discovered, he suggested that cybersecurity may need a fundamentally different defensive model, potentially using advanced AI itself.

Watch Sam Altman’s Stanford interview — cybersecurity discussion begins around 19 minutes

I found that particularly significant.

Because I’ve been thinking about a closely related problem, although from a somewhat different perspective.

Altman is questioning whether our existing approach to defending software can keep up.

I’m questioning whether the architectural assumptions underlying modern software security need to evolve as well.

These aren’t identical arguments.

And Altman’s warning doesn’t prove my theory.

But it reinforces the importance of asking whether security processes designed around comparatively slower discovery and response cycles can withstand increasingly capable automated adversaries.

What happens when finding vulnerabilities becomes significantly faster than evaluating, fixing, testing and deploying the necessary security updates?

That’s the question I believe the industry needs to confront.

October 2026 Is Already Providing Evidence

This isn’t purely a discussion about what might become possible five or ten years from now.

Look at what happened during the week of October 19–23, 2026.

PHP: Two critical vulnerabilities were disclosed, each rated 9.2 out of 10 under CVSS 4.0. Under vulnerable conditions, exploitation could allow unauthorized code execution or exposure of sensitive memory. OpenAI documented how AI-assisted research contributed to discovering one of these vulnerabilities.
Node.js: Security releases addressed several high-severity vulnerabilities, including a memory-corruption flaw that could potentially permit remote code execution. The Node.js project credited OpenAI Codex Security with discovering that issue.
Ruby and GitLab: Ruby released updates fixing flaws in its regular-expression engine that had contributed to serious vulnerabilities in GitLab, including an issue previously rated 10 out of 10.
Java and certificate infrastructure: A compromised certificate-authority incident required security responses across major software ecosystems, including an Oracle Java security update.
Python software ecosystem: Attackers compromised LiteLLM’s release pipeline, distributing a malicious package designed to steal credentials and enable unauthorized code execution.

These are different types of security incidents.

Not all were discovered by AI. Not all were successful attacks. And the existence of a vulnerability does not mean every system using the affected technology was compromised.

But consider the technologies involved.

PHP and Node.js are used extensively to power internet applications.

Java is deeply integrated into enterprise computing.

Ruby underpins important software platforms.

Python packages are used throughout automation and AI development.

And digital certificates help establish trust between systems communicating over the internet.

We’re discussing vulnerabilities and compromises affecting technologies that form part of the foundations of modern computing.

What surprises me, even after 16 years of working in software, is how deep some of the AI-assisted discoveries are reaching.

And my concern isn’t limited to the vulnerabilities that have already been disclosed.

It’s about what happens as these discovery capabilities improve.

AI Is Already Testing the Security Boundaries of AI Itself

On October 23, OpenAI published technical details about how it has strengthened security around its frontier-model training infrastructure.

In authorized internal research, fleets of advanced AI agents were instructed to investigate the infrastructure and test whether they could cross security boundaries.

According to OpenAI, those experiments uncovered previously unknown software vulnerabilities and configuration weaknesses.

The agents demonstrated ways to escape virtual-machine isolation, move across security boundaries and reach restricted network resources under certain experimental conditions.

The company subsequently strengthened its infrastructure, repaired vulnerabilities and expanded its defensive testing.

This doesn’t mean an ordinary person can instruct any current AI model to compromise any computer.

These were specialized experiments performed under controlled conditions with substantial resources.

But they demonstrate something important.

Advanced AI agents are already capable of discovering unexpected weaknesses in complex security environments.

And this strengthens the case for examining whether our defensive assumptions need to change.

Read OpenAI’s technical report: Securing Our Frontier Model Training Environment

Why I Believe Security V2 May Become Necessary

For me, the most important question isn’t whether AI will discover more vulnerabilities.

I believe it will.

The harder question is what kind of software architecture can remain secure when increasingly powerful automated systems are constantly looking for weaknesses.

Historically, many security strategies have relied on identifying vulnerabilities, deploying patches, monitoring suspicious activity and preventing unauthorized access.

These practices remain essential.

I’m not suggesting that patching software will become unnecessary.

But what happens when the pace of discovering vulnerabilities increases dramatically?

What happens when attackers don’t just target one application, but systematically investigate its programming language, dependencies, authentication systems, deployment infrastructure and cloud services?

And what happens when several seemingly minor weaknesses can be discovered and combined automatically?

I suspect we may need an architectural approach that assumes vulnerabilities will eventually be found, rather than relying primarily on preventing their discovery.

Some of the principles already exist.

We have isolation, sandboxing, least-privilege permissions, zero-trust architecture, defense in depth and many other established security practices.

But I believe the way we apply these principles may need to become substantially more rigorous.

For example:

A vulnerability in one component should not automatically provide access to unrelated systems.
A compromised application should have extremely limited access to credentials, infrastructure and sensitive information.
AI agents should operate within enforceable permissions and security boundaries, not merely instructions telling them what they’re allowed to do.
Security analysis may need to become continuous, with defensive AI agents inspecting systems as frequently as attackers might.
Organizations may need the ability to detect, verify and deploy security updates far more quickly than traditional engineering workflows allow.

None of these ideas is entirely new.

But their importance changes when sophisticated vulnerability discovery becomes increasingly automated.

Security V2, as I imagine it, is not one new programming language or operating system.

It’s a broader evolution toward software systems that can remain resilient even when powerful adversaries are capable of discovering and exploiting weaknesses at unprecedented speed.

And importantly, AI itself may become one of the most effective tools for building those defenses.

The Other Concern: Powerful AI May Become Accessible to More People

There’s another aspect of this that I believe deserves attention.

The risk is not limited to the capabilities of the most advanced models today.

We also need to consider what happens when increasingly capable systems become widely accessible.

Some commercial AI systems include safeguards, monitoring and restrictions intended to reduce misuse.

But open-weight models can sometimes be modified and operated on infrastructure outside the original developer’s control.

These models provide substantial benefits for innovation, research and competition.

However, their safeguards may also be more difficult to enforce once the models are distributed.

This does not establish that open-weight AI caused the security incidents discussed here.

Nor does it mean every open model is capable of sophisticated vulnerability discovery.

The concern is what becomes possible as the technology advances.

We don’t necessarily need dramatically more intelligent models for the security landscape to change. We may also see major changes as people learn to combine existing models with increasingly powerful tools and coordinated agent systems.

Capabilities that once required significant resources could become more accessible.

And some people will inevitably attempt to use those capabilities maliciously.

This is one reason I believe the architecture itself needs to become more resilient.

Why the Next 30 Days Concern Me

On October 22, Anthropic’s deputy Chief Information Security Officer, Jason D. Clinton, warned that cybersecurity conditions could become significantly worse before improving.

He suggested organizations might face approximately 18 months of difficult security challenges.

Marc Benioff, Salesforce’s CEO, publicly supported the warning.

I agree with the broader concern.

But I’m looking at a shorter timeframe.

I believe the next 30 days, from October 23 to November 22, 2026, could be extremely challenging for the software security industry.

I expect more significant vulnerability disclosures, emergency security updates and attempts to exploit newly revealed weaknesses.

I also suspect we’re going to learn more about security vulnerabilities that existed for years without being publicly identified.

That’s my prediction, not an established forecast.

And I genuinely hope I’m wrong about the severity.

I’m not predicting that the internet will collapse or that critical services will necessarily fail.

But the amount of security work potentially ahead of us is considerable.

Especially when we remember that publicly disclosed vulnerabilities are only the ones we know about.

There may be other weaknesses that haven’t yet been discovered or reported.

Some attackers may already possess information about vulnerabilities defenders don’t know exist.

That’s always been a possibility in cybersecurity.

AI could make that challenge considerably more difficult.

Software Engineers and Security Specialists May Be Needed More Than Ever

There’s something else I find interesting about this situation.

We’re constantly discussing how AI may replace software engineers.

But consider what happens if advanced AI substantially accelerates vulnerability discovery.

Someone still needs to understand the infrastructure.

Someone needs to determine whether an application is exposed.

Someone needs to develop or validate fixes.

Someone needs to understand the consequences of updating a dependency in a complex production environment.

And someone needs to ensure that the proposed solution doesn’t introduce additional problems.

AI can already assist with many of these tasks and may increasingly automate portions of them.

But production software exists within real businesses, with real dependencies, customers, data and operational consequences.

Finding a vulnerability and safely fixing it across millions of deployed systems are two very different challenges.

In the near term, I suspect experienced software engineers, cybersecurity professionals and infrastructure specialists may face an exceptionally busy period.

And I believe strong engineering judgment will remain important, even as AI changes how much of the work is performed.

I Hope Our Software Infrastructure Holds Up

I want to end with something important.

I’m extremely optimistic about artificial intelligence.

I use AI extensively in software development, and I believe its potential benefits are extraordinary.

But optimism doesn’t mean ignoring the risks.

And after watching what happened this week, I genuinely hope that technology companies, security researchers and engineering teams can respond quickly enough.

Because we’re not just talking about small websites or isolated databases.

We’re talking about infrastructure that supports banking, financial transactions, healthcare, communications, businesses, government services and everyday technologies most people never think about.

People simply expect these systems to work.

And they should be able to.

I hope security researchers continue finding critical vulnerabilities before malicious actors exploit them.

I hope AI becomes an extraordinary advantage for defenders.

And I hope the next 30 days prove my concerns too pessimistic.

But long term, I believe we need to take the architectural question seriously.

For decades, software security evolved around a world where human capabilities helped determine the limits of sophisticated attacks.

We’re entering a world where increasingly capable AI agents may fundamentally change those limits.

My theory is that the software industry will eventually need to evolve from what I call Security V1 to Security V2.

Not because everything we built previously was wrong.

But because the threat environment may be changing faster than the systems designed to protect us.

Sam Altman’s warning reinforces that concern.

And the developments of October 2026 suggest the transition is no longer a purely theoretical discussion.

Security V1 was shaped by human-scale threats.
Security V2 must be ready for machine-scale threats.

I hope we get there before serious damage forces us to.

Sources and Further Reading

This article presents my personal perspective as a software engineer, not a formal cybersecurity threat-intelligence assessment. Software Security V1 and V2 are my own conceptual terms. The future risks discussed are predictions, not established outcomes.

About the Author

Marian Sorca is a software engineer and founder of abZ Global and SelfManager.ai. He has worked in software development since 2010, delivering custom websites, business applications and SaaS platforms for clients, primarily in the United States. His technical experience includes frontend and backend engineering, cloud infrastructure, application architecture and AI integrations.