Magic, or just a lack of understanding

When a magician pulls birds out of thin air, makes objects vanish, or turns one card into another, we are often left in awe and confusion. Some tricks are so mind-bending that the magician almost seems to have supernatural powers.

Behind the scenes, though, magicians rely on tricks like exploiting human psychology, misdirection, and optical illusions. In the moment, it truly feels magical, but once the trick is revealed, everything makes sense and it suddenly seems obvious.

Similarly, many of the things we do intellectually, such as hacking software, composing music, and generating new ideas, can seem like magic at first. That points to a gap in our understanding of how the brain works, and scientific inquiry can often bridge it.

The old philosophical riddle about whether a tree falling in a forest with no one to hear it makes a sound stops being a mystery once we understand sound as vibrating air waves.

The same applies to consciousness and creativity. They may feel magical now, but I expect that with time the mystery will dissolve.

Intelligence, by contrast, already has several hypotheses explaining how it works. There’s no complete theory of every part yet, but current cognitive science and neuroscience suggest that it is modular, distributed, hierarchical, and predictive, emerging from the interaction of systems like perception, memory, decision making, learning, and attention.

Marvin Minsky’s Society of Mind is one theory that is particularly useful for building AI agents, and it fits both cognitive science and modern artificial intelligence surprisingly well.

Minsky proposed that intelligent behavior can be broken down into simple components, or “agents,” that each handle a narrow task. These agents interact and cooperate in hierarchical structures to produce intelligence.

Even if Minsky’s framework isn’t entirely accurate about how real minds work, IMO treating hacking as the work of a specialized sub-agent, and narrowing down what that agent does, is still useful for building more effective, targeted agents.

The hacking agent in my brain

When I am not hacking, I find great joy reading psychology, philosophy, and the study of consciousness, intelligence, and creativity. Exploring these questions has always made me feel more alive. AI made this even more exciting, because I can now explore hacking and intelligence together by building hacking agents.

For any given task, the frame of agency changes, and the agent has to change with it. A “coding agent” operates completely differently from a “hacking agent.” The goals and strategies are different, so each agent needs its own hierarchy and heuristics for approaching the goal. (Switching between the coding agent frame and the hacking agent frame could be the sole reason for many software security issues.)

To develop a hacking agent, I first need to understand how the hacking agent in my own brain works, and how it generally operates for top-tier hackers. Only by understanding that process can I begin to transfer it into an AI agent.

Formalizing hacking and the science of insecurity

To create capable AI hacking agents, we must approach insecurity from first principles. In my view, the root of insecurity is not negligence but the cognitive limitations of humans: our difficulty reasoning about trust in deeply composable software systems, and the overwhelming complexity of software state machines.

A hacker’s job is to understand that complexity at many levels, understand it better than the original author, and break the assumptions holding the system together.

At a fundamental level, software systems are finite state machines. Each instruction or input transitions the system from one state to another. And we use Turing-complete languages to implement these state transitions in real-world code.

Take a simple login system, for example. Its high-level state path might look like this (of course, each of these states may include their own internal substates and branching logic):

 -> INIT -> username and password entered -> password verified -> authenticated

A programmer’s task is to write such state machines, embedding the logic that governs how transitions occur. That logic is the decision-making layer that controls state transitions, and Turing-complete languages give programmers the expressive power to encode it, for browsers, web apps, protocols, and beyond.

In a perfect world, programs would do exactly what they were designed to do, and there would be no malicious actors to tamper with them or coerce them into weird machines. Security would be unnecessary and trust would be implicit. We don’t live in that world, which is why we need hackers to create trusted systems before criminals exploit them.

A hacker’s job is to explore a system more deeply than the person who built it, understanding it at multiple levels, mentally reconstructing its state machine, and breaking its logical assumptions.

Limits of security and the danger of AI

Even world-class hackers are bound by cognitive limits. We can’t reason about all possible states, branches, and interactions in composable, dynamic systems.

If we ever witness an intelligence explosion (AI systems that can reason far beyond human cognitive limits), it’s likely they will discover more weird machines than we ever could. Exploits that are invisible to the human mind may become trivial for them, so such AI capabilities must not fall into the hands of malicious actors.

Even so, superintelligent AI would face hard theoretical limits on achieving perfect security (Rice’s theorem).

Vulnerability discovery as theorem proving

At a high level, vulnerability discovery is a logical problem. The goal of a hacker, or of a security reasoning system, is to construct a proof that a program’s behavior can violate its specification or assumptions. This can be done by guessing or brute-forcing, but it usually happens by systematically identifying a logical inconsistency in the system’s structure.

This idea is grounded in the Curry–Howard correspondence, which draws a direct parallel between computation and logic. Programs are proofs, and proofs are programs. Running a program is akin to constructing a logical argument. By extension, vulnerabilities are proofs too. They arise when a hacker constructs a logical path (i.e., an exploit) that demonstrates a proposition such as “the user input reaches a certain sink.”

In this view, vulnerability hunting, whether by humans or by AI agents, is adversarial theorem proving, where the proposition to be proven is:

That unintended behavior might look like:

• Can untrusted input reach a dangerous sink?

• Can the attacker elevate privileges?

• Can this sequence of states bypass assumptions?

These are all questions about the semantic properties of programs (their meaning, not only their syntax or structure). That said, most of the vulnerabilities developers introduce in web applications are syntactic issues that current AI agents or SAST tools can easily find.

But what separates top-tier human researchers is their ability to reason about semantic vulnerabilities: issues that emerge from deep interactions across various systems, state transitions, and multi-layer logic.

Not all humans think alike

For me, the difference in hacking ability between people becomes especially clear in CTF competitions, where some of my friends process complex systems and identify vulnerabilities remarkably fast. In my opinion, this isn’t always something you can learn through practice alone. I’ve seen newcomers, people with almost no background in hacking, rapidly outpace others simply because of how quickly they digest information and reason.

This suggests that effective vulnerability discovery in complex targets requires a particular mix of cognitive abilities, which I’d generalize into three:

1. Long-term memory stores and recalls core concepts, prior vulnerabilities, bug classes, system behaviors, and high-level heuristics. This enables variant analysis and generalization.

2. Working memory is a mental “scratchpad” that actively holds and manipulates details of the current subsystem or bug candidate. The larger the working memory, the more state and logic one can juggle simultaneously.

3. Processing information is the engine behind reasoning. It allows the hacker to simulate hypotheses, test edge cases, and compress raw inputs into abstract representations. Even with strong memory, slow processing yields weak results. Conversely, high processing speed enables rapid internal updates and better state manipulation. I also think good processing helps store information well in long-term and working memory.

One aspect of memory is particularly relevant for AI agents. Even top-tier human hackers never ingest full codebases into working memory. Instead, they construct and rely on specific abstract representations, some kind of control-flow skeleton. These abstractions allow them to compress vast complexity into something they can actually reason about.

One of the areas I’m actively researching is how to replicate this ability in AI agents, by extracting the right abstractions from large codebases, storing them meaningfully, and performing targeted reasoning over them to find vulnerabilities.

Two modes of vulnerability discovery

Known propositions (variant analysis)

This is the majority of modern vulnerability discovery: looking for known bug classes, such as SQL injection, SSRF, XSS, and authn and authz issues, in new code. Rather than inventing new propositions, you’re just proving them in a new context.

This form of analysis is currently the primary focus of Hacktron, and it accounts for 60–70% of real-world web application vulnerabilities. The search starts with a known vulnerability class, retrieves its defining characteristics and exploitation patterns, and then systematically processes the target codebase to determine whether that pattern exists. Realistically, we expect to solve this at Hacktron if everything works out.

However, variant analysis in complex systems like V8 is much harder. These bugs often depend on deep contextual understanding of specific subsystems.

Novel propositions (new research)

Novel research, in contrast, involves generating new propositions based on a deep understanding of system behavior. These are often semantic, logic, or design-level bugs, and they require:

• A thorough understanding of the subsystem, which itself demands significant processing power to reason through complex architectures.

• Creative capacity to hypothesize how things might go wrong and break the trust.

• Sufficient computational bandwidth (mental or machine) to simulate those hypotheses and explore potential weird states.

This is what I aspire to solve at Hacktron, because I’m not great at this myself. I’m envious (in a good way) of people who can dive into V8 internals or kernel subsystems and find deep, structural flaws. So instead of trying to become one of them, I want to build something that comes out of me, an AI system that can do that.

Case study: hacking V8 via WASM

A CTF teammate of mine once hacked V8’s WASM type system and landed a full exploit chain within just three days. That’s one of the wildest things I’ve ever seen pulled off.

Here’s my rough (and probably naive) understanding of how he did it:

First he had to understand the system, the job of the understanding agent. He didn’t start with deep knowledge of V8’s WASM internals, but he had strong fundamentals in type systems, compilers, and how computers work at a low level. From there, he dove into WASM specs, RFCs, and blog posts. Even parsing and internalizing those large specifications in a day or two requires what feels like superhuman processing speed.

With that kind of cognitive throughput, you can build efficient internal representations of complex systems. Some people have this ability, not to memorize everything but to mentally structure complexity so that reasoning about it feels almost intuitive. Once he understood the core concepts, like WASM’s isorecursive type system, he built a compressed internal model of it, and this could happen in parallel. It probably wasn’t the full source code in his head but something abstract, maybe a control-flow diagram or a conceptual graph of how types are resolved and passed through the engine. The quality of that representation, how well its structure captures essential behavior, matters more than raw memory.

Then he had to manipulate the system, the job of the reasoning agent. This is where the hacker mindset agent comes in, and the frame has to change to simulating what could go wrong. What happens if a type check fails? What happens if an optimization assumes something that turns out to be false? He ran these “what-if” computations mentally, probing the gap between the type system as designed and as implemented. That’s where vulnerabilities live.

Even V8 developers might not have this kind of adversarial model in mind, because they’re focused on building, not breaking. A hacker has the freedom to focus purely on failure, and whatever assumptions the system was built on become targets.

Why this matters for Hacktron

You don’t need to load the full codebase to find deep bugs. An agent needs to:

• Navigate and isolate a subsystem

• Build an internal model or abstraction of how it works

• Simulate hypotheses and see what breaks

• Compress that insight and hand it off to a vulnerability-finding agent

That’s how humans do it, and maybe AI should be modeled the same way. I think this is how AI will eventually hack large, complex systems like V8, and it’s what we’re trying to build.