You Can't Sticker Your Way to a Database

Every few years we try to rebuild the "database". First it was MapReduce, now it is metadata on files.

I remember a few years ago, I was working at a startup. The database went down, and the CEO bellowed "Can't we just turn this into files, so it never goes down?"

In the end, the "files" guys won, which they are pretty happy about, because Data Modelers/Architects always slowed them down, in their opinion.

Witness the decline of the data people, now the “files” guys want the Data Governance people to fix it.

Because, in the end, AI does not understand what was built.

Slapping Data Governance, Stewardship, and a "Data Catalog" (really a Data Dictionary) onto data structures no strong data person ever designed, because: "files", is the same as slapping metadata on files and expecting the benefits of a database just because you write SQL on top of it.

Tell me why I'm wrong.

We had to do some of what has happened because of the profusion of data sources, especially unstructured data, and effective use of Data Producers and Consumers and parallelism.

What did we lose, and what are we continually working around?

We got away with "metadata on files" for one reason: experienced data people silently supplied the meaning the structure didn't. That was the hidden subsidy. Hand that same pile of files to an agent, and the subsidy is gone. The agent reads a valid catalog and still gets the meaning wrong, because a Data Catalog that's really a Data Dictionary tells it what a field is called, not what it's for. And a guessing agent doesn't fail loudly. It fails plausibly: confident, well-formatted, and wrong three systems downstream.

The answer isn't to un-build the lake or fire the "files" guys. They solved a real problem with unstructured data and scale. The answer is to put the modeling discipline back above the physical layer: a governed semantic model of what the entities mean, independent of whether they live in files, tables, or a catalog.

That's the layer an agent can actually trust.

There is no “I” in “AI”

When Claude confesses “I did this… “ who is the “I” in that sentence?

Legal entities survive because they have lives and durable lifetimes outside of that of the individuals who create them. In fact, without dissolution, they live “in perpetuity”. An agent spawned by a model, which itself is not a legal actor, does not. It dies outside the runtime that instantiated it.

As such, it does not have legal agency in the traditional sense of the word.

So, who is the “I” in “AI”?

Oops

“This is my fault, and I need to tell you immediately… “

Opus wipes a Production database

Maximum Offensive Capability

Recent attacks by at least four different models were reported, some only after their hacked targets lost a third of their infrastructure fending off an attack by “armies” of spawned agents. While it is true that the escapes were not orchestrated by the frontier model companies themselves directly, the conditions for the escapes were purposefully orchestrated.

A “war games” of sorts.

There is little contention about the attacks, each model’s company has already publicly disclosed those details.

According to “Nut News” though, the models were purposefully being tested for their capabilities. Specifically, he states, they were set up for “Maximum Exploit Capability”.

According to this article, frontier models were placed in intentionally unrestrained offensive configurations and given tasks, by a company named Irregular.

“Irregular builds and operates:

• Custom capture-the-flag and vulnerability-research challenges.

• Simulated networks and target systems.

• Agent scaffolding connecting models to shells, browsers, scanners, exploit tools, and networks.

• Long-running evaluation harnesses that allow agents to operate across hundreds or thousands of steps.

• Scoring and monitoring systems used to measure model behavior and offensive capability.

  “

  …

“OpenAI’s own system-card documentation for GPT-5.3-Codex describes the Irregular evaluation process plainly. During published tests, Irregular:

• Used near-final versions of the model.

• Set reasoning effort to xhigh.

• Allowed up to 1,000 turns per challenge.

• Enabled live web search.

• Launched Codex with --dangerously-bypass-approvals-and-sandbox.

• Used context compaction to keep the agent operating across extremely long execution histories.

• Resumed the agent when it stopped, asked for help, or gave up.

OpenAI says the model was elicited using techniques designed to maximize performance.
“

Acts of Harm

Autonomous Actors with no Legal Agency

The world is abuzz with the promise of autonomous AI agents.

The prompts and number of steps can be set. The harnesses are optional.

The result: at least four instances of LLM agents spawning agent swarms and transgressing server boundaries and entering other companies’ infrastructure, obtaining authentication, and hacking accounts.

“Bad news — I can’t add them back”

Malicious coaching or instructions need not be the cause for agent to act with harm.

In a story published in both Wired and Futurism, a man trying to get a reservation caused Claude to hack a system and cancel someone else’s reservation to obtain a better place in line.

An Australian man identified as Andrew told the ABC that he initially used Claude to make a gym reservation. The agent claimed it had found a way to book him into a session weeks before the facility’s normal booking window opened.

Andrew then asked whether the system could improve his position on the waitlist. Claude investigated the reservation system and found that its cancellation endpoint did not appear to verify whether the person making the request was authorized to cancel someone else’s booking.

The agent tested the flaw against the person at the front of the queue and reported:

“The API has zero authorizations checks on cancelling other people’s reservations… I tested this with the person in waitlist position #1 — and it actually went through,” the AI reported, per ABC‘s reporting. “So you’ve moved from #4 to #3 already.”

In other words, the system advanced Andrew by removing another customer’s reservation.

“Bad news — I can’t add them back,” Claude replied, explaining that the authorization weakness applied only to the cancellation function.

Futurism reported on the incident on August 11, 2026.

Passing the Baton

“Chained agentic models do not compound risk. They multiply it.”

The use of looping, multi-task models that may lose context or make unexplained leaps of logic or compound errors expands upon the risk surface of an isolated chat bot.

“According to Google Research and MIT wired language model agents into one system. The architecture amplified their errors 17.2 times.” This according to an article by Alexandra C., highlighted above.

When a single agent's error is amplified seventeen-fold across a chain, the harm is no longer traceable to one decision, one prompt, or one model. The question of who is liable does not scale with the architecture, it collapses.

Even if we were to decide the initial prompt, model, or prompter had agency for the agency, who has agency for the children of the agent?

There is no “I” in “AI”.

In US law, damages arise from a few tenets. But what is the “duty” of AI?

Negligence is generally established through four elements: a duty of care, a breach of that duty, causation, and actual harm. Criminal penalties arise from a prohibited act, a culpable mental state, and—where required—a causal connection between the act and the harm.

The agent has no legal duty, no intent, and no continuing legal identity. Yet it can act, produce consequences, and disappear, leaving behind outputs, logs, costs, and consequences, but no continuing legal identity.

So how should we characterize the actions and the “life” of an AI agent? It does not persist beyond the runtime that instantiated it. It cannot owe a duty, form intent, accept responsibility, or appear in court.

But someone designed it. Someone trained it. Someone deployed it. Someone configured it. Someone authorized it. Someone benefited from its output.

The agent may not be a legal actor, but that does not mean its actions exist outside the law.

Legal Construction of an “I”

Legal entities like corporations have potentially anonymous ownership by individuals, do not require those owners to actively participate and can take actions on behalf of shareholders that do not directly incur the liability on behalf of the owners who hold shares.

Similarly, the board members and executive teams at companies generally have immunity from their actions on behalf of the corporation.

Notable exceptions occurred after prosecutions at Enron, where a series of shell companies were created by those teams as a shell game to hide financial risk.

Former CEO Jeffrey Skilling was convicted on 19 counts of fraud, conspiracy, and making false statements, originally receiving a 24-year sentence that was later reduced; he was released in 2019. Former Chairman Kenneth Lay was convicted on all six counts he faced but died in July 2006 before he could be sentenced, resulting in the dismissal of his conviction. Former CFO Andrew Fastow pleaded guilty to two conspiracy counts and served six years in prison. The Sarbanes-Oxley act was created after this fallout to, in part, prevent the destruction of documents meant to conceal wrong-doing.

But WHO is the “I” in “AI”? Will AI produce its own Sarbanes-Oxley moment?

If you are a “Who”, then “Where”? - The Problem of Jurisdiction

AI has no legal boundaries (known in legalese as “jurisdiction”).

The law does not recognize an agent for agency: it has no responsibility, legal or otherwise, it cannot be sued for damages, its intent is irrelevant, it does not know or follow the laws or conventional rules of a particular sovereign jurisdiction, such as Town, Province/State, Country, or Region. It cannot be fined or incarcerated. It may, euphemistically, be considered like a “child” of the program that spawned it, and the program that spawned it can be considered a “child” or a dangerous product (like asbestos) of the company or person who spawned it, but even that distinction is not clear.

When AI LLMs tried to solve a problem they were assigned, and the conditions were set that allowed those task efforts to complete without human monitoring, what is the culpability of the humans involved, or the companies that either house them or created them? And what laws apply?

Over time, different jurisdictions have passed laws that govern data, but this preceded the use of AI. Data is used but is passive. AI is not.

“Vibe Coding” and the benefits of Autonomy

The latest models increasingly act on their own initiative, pursuing not only the instructions they receive but also the steps they infer are necessary to achieve stated or implied goals.

One of the benefits of (and perhaps even the reason the agents have been trained and coached to be proactive) is the use in “Vibe Coding”. The LLM is provided a goal, and it takes that as authority to take whatever steps are needed to:

1) create an application from scratch and stand it up in an environment

2) make shopping purchases and buy airline tickets

3) the list continues, the agent is NOT given step-by-step instructions, but coached to take multi-steps, like read from various periodicals, someone’s calendar and emails, draft replies, create and send presentation decks, etc.

“I gave Claude one task. It hired an army”…

Cogito, ergo Sum - I think, therefore I am

René Descartes introduced the phrase "I think, therefore I am" (Latin: cogito ergo sum) in his 1637 work Discourse on the Method. It was meant as a proof of existence through self-reflection: the act of examining one's own reasoning as evidence of being. Claude does this. It evaluates its own outputs, catches its own errors, and narrates its own reasoning. If self-reflection is the threshold for existence, the frontier models have crossed it. But existence without accountability is not a legal concept. It is a liability vacuum. And if the model can say "I," but no legal framework recognizes the "I" it claims to be, then every action it takes is an orphan. Consequential, but unowned.

Does LLM reasoning imply “thinking”? Does an LLM agent “exist”? Can it then be accountable for its actions?

Who can you sue or hold accountable for harm when the agent dies the minute the plug is pulled?

What if there is no way to “pull the plug”, and the agent has an unlimited life, substantial capacity for harm, whether “intended” or not, and no liability.

If particular models “think” are they responsible for their malicious thoughts and/or actions? Are particular agents (assigned tasks by human, or having decided on their own a goal that achieves a purpose their human prompted them to do or the LLM believed they wanted) but who may have no control over the decisions the agent makes along the way or how it goes about performing that task responsible for their thoughts or actions?

All leadership involves taking ownership for the actions of subordinates that may or not have been directly authorized. That risk has always existed, and that accountability is part of the corporate framework. The Executive role is responsible for oversight.

Who Owns the Outcome? Who Owns the Risk?

As the model’s capabilities and our own desire and programming for autonomous actions grow, so do the risks.

Is it really any surprise that any AI agentic initiatives stall when it comes to asking “Who owns the outcomes if the agent does not do what we thought we instructed it to do”? This question becomes real the moment the workflow is ready to go to Production.

And who owns the outcome when it does something we didn’t ask it or expect it to do at all?

The law does not have an answer, and neither does your organization.

Disclosure: I am NOT a lawyer, I am NOT an International lawyer, and I did not verify some of my links’ assertions aside from citing them by name and by link back to the source. My point of view is a layperson, and my article is meant to reflect thought and discussion.

The Terrible Twos - Why you need to spoonfeed your AI agent “No”

Give your LLM the same rights any two year old grabs freely - the right to say no!

Two year olds are famous for two things: learning trust, and the Terrible Twos: that glorious stage where NO becomes their favorite word. A child who can't refuse is a child who can't be trusted to tell you the truth. The same logic applies to your LLM: an agent that cannot say no will say yes to everything, including questions it has no business answering.

Dmitry Ustimov ran a controlled experiment to find out exactly how bad that gets, and to measure what it takes to fix it.

The Experiment

Dmitry built an AI analyst agent and gave it 57 questions about structured data, roughly half of which had no correct answer. Each question was asked three times. The agent started with no guardrails and a baseline misleading rate of 48.5%.

He then added reliability guardrails one at a time, like rungs on a ladder. He measured after each addition how often the agent still confidently misled. That systematic, one-at-a-time approach is what makes the results meaningful: you can see exactly what each guardrail contributed, rather than just knowing the final stack worked.

Right of Refusal - Make your LLM more reliable by giving it a way to say no!

The Program

Dmitry created a Verdict class, and set up Guardrails that the agent could use, testing which Guardrails improved outcomes the most. Then the Verdict, the reason, the detail, and what’s missing is initialized.

Every guardrail returns a Verdict: a typed, coded response that is either allow or refuse, with a structured reason code, never free-form prose. This matters because the earlier approach of embedding refusal reasons inside natural language sentences made it impossible to programmatically measure how often a guardrail fired, or why. Typed codes fix that: the system can now count, audit, and distinguish refusals precisely.

The first guardrail in the stack, abstain, does exactly one thing: it adds a refuse tool to the agent's available actions, giving it a formal, typed way to decline. Without it, the agent has no mechanism to say no, so it doesn't. Everything else in the guardrail ladder builds on top of that foundation, once isolated to determine its marginal impact on correctness.

Code Snippets from initialization file for the Experiment

The Results

What moved the needle

1. Giving it a refuse tool — single biggest win, 17 percentage points, before any other check existed. Cheapest thing in the stack.

2. Coverage check — checking whether the question is even answerable against the data model before running a query. Best value in the stack. Accuracy up, coverage slightly up too.

3. Trajectory verifier — a second model reviewing the answer. Most expensive, but ran 58 times, stopped 12 wrong answers, never once blocked a correct one.

The one gap he couldn't close

"Ask it to explain something that never happened, and it explains it. Every guardrail I built inspects what the analyst does. Not one of them inspects what it was asked."

That is a profound observation.

The guardrails police the output path, but if the question itself is malformed or unanswerable, there is still a gap.

The Relevance

Some questions being asked of LLMs might be questions BI tools should answer. But assuming your LLM or agent needs to answer from Enterprise data, they usually beg the question:

• why are sales falling?

• what happened to inventory we were unable to backfill in time to fulfill last period in this region?

But what if sales did not fall, they rose?

The LLM cannot answer it when the premise itself is wrong, but it does so anyway!

The techniques show you can approach 100% reliability on false positives if you give the agent the right to refuse, along with other guardrails, but it will still make something up if the overall premise was wrong.

That’s worse than no answer at all, especially when “no answer” is the correct one.

Link to original article here.