Security, Machine Discoverability, and the Cost of Legitimate False Negatives

The first two articles in this short series began with a simple experiment: I asked several AI systems to inspect OSI-PIA-clean, a public GitHub repository for Organizational Systems Intelligence and Professional Identity Architecture.

Repository: https://github.com/Dingus-McGuillacutty/OSI-PIA-clean

The experiment produced several different machine behaviors. Kimi K3 could not reach the repository and abstained from describing it. Gemini Flash-Lite could not reach it and partially filled the gap with an incorrect guess about what “OSI” meant. Grok Fast reached the material and reconstructed much of the project, but initially relied on a designated current-status document whose temporal authority had gone stale. Claude Sonnet 5, in the environment tested, reported that it could not fetch either the GitHub repository or its GitHub Pages site because those sources had not first appeared through the search pathway available to its tools.

That last failure became the starting point for The False Negative Stack. The repository was real, public, and directly reachable by a human browser, yet it never entered Claude’s context. The source had not failed a substantive evaluation; it had failed the machine’s recognition and admission pathway.

Further investigation changed the interpretation in an important way. Part of the problem was mine.

Testing note: These observations describe specific products, model-and-tool environments, search results, and dates tested. They do not establish permanent capabilities or limitations of the underlying models or vendors.

Public is not the same as discoverable

After the Claude test, I tried to find my own repository through ordinary open-web search. I could not reliably surface it. Independent searches conducted during this investigation likewise failed to return OSI-PIA-clean for obvious project-name and repository-name queries.

That is an ordinary discoverability problem, and it is something I can work on. A public URL can exist, function normally, and still be poorly indexed. For a human who already has the exact address, that distinction may barely matter. For a machine environment whose retrieval tools depend on search-mediated discovery, it can matter completely.

The chain is therefore more precise than “Claude could not read GitHub”:

public to humans
→ discoverable through available search
→ source-admissible
→ machine-reachable
→ machine-readable
→ machine-understood

OSI-PIA-clean was public to humans and directly reachable by URL. In the Claude environment tested, that was not enough. The source also needed to satisfy an upstream discovery condition before the fetch path would proceed. That means the earlier false negative was not evidence that the repository was defective, nor evidence that Claude was incapable of understanding it. The source never reached the point where understanding could be tested.

Then the threat model became clearer

The next piece came from Anthropic’s September 2026 threat-intelligence report, Detecting and countering misuse of AI: September 2026:

https://www.anthropic.com/threat-intelligence-report-september-2026

Anthropic reports that, between December 2025 and August 2026, its threat-intelligence team identified and disrupted malicious uses of Claude across cyber operations, surveillance, influence operations, scams and fraud, biological misuse, conventional weapons development, and illicit model distillation. The report describes sophisticated actors repeatedly testing safeguards and attempting to circumvent technical controls.

Some examples are directly relevant to source and task admission. Anthropic describes threat actors who concealed the larger purpose of their work, distributed activity across many sessions so that no single interaction revealed the full intent, used Claude in cyber operations across multiple stages of attack activity, and used Claude in weapons-development workflows. In one conventional-weapons case, Anthropic reports that actors explicitly hid what software was for and split work across multiple sessions to evade safeguards.

The report does not say that the exact search-before-fetch behavior I observed in Claude was created because of these incidents. I have no evidence for that causal claim. It does, however, establish that Anthropic operates in a real adversarial environment where users may deliberately disguise intent, fragment tasks, supply external material, and attempt to use computational tools in ways the provider is actively trying to detect and prevent.

Under those conditions, a conservative boundary around arbitrary user-directed external sources is not difficult to understand.

A reasonable gate can still reject the right thing

This changes the lesson from the original experiment. The interesting conclusion is not that Claude had a bad or foolish gate. The more interesting possibility is that the gate may be quite rational.

An AI system with tools capable of searching, fetching, reading, coding, and acting across external systems has a much larger attack surface than a text-only chatbot. A user-supplied location might contain ordinary documentation, but it might also participate in prompt injection, social engineering, credential theft, malicious code, poisoned instructions, staged cyber activity, or a larger task whose dangerous purpose has deliberately been hidden from any single interaction.

A provider facing that environment has reasons to be conservative about what enters the system’s trusted working context. Security engineering routinely accepts some inconvenience and some legitimate denial when the expected cost of a bad admission is sufficiently high. But the cost on the other side does not disappear.

legitimate public source
→ poor search visibility
→ source fails admission path
→ model never inspects content
→ legitimate false negative

The gate can be justified and the result can still be wrong. That is the systems problem.

Security has always lived with both sides of the error

Security controls are often discussed as though their job is simply to stop bad things. In practice, every control creates an error surface. Tightening a control may reduce false positives of trust — malicious things being admitted as safe — while increasing false negatives — legitimate things being denied because they cannot satisfy the preferred trust path.

Mature security systems therefore do not depend on one binary question: trusted or rejected? They create additional states and additional procedures. A source may be unknown rather than malicious, unverified rather than false, constrained rather than fully trusted. A failed primary verification path can trigger additional authentication, sandboxing, limited permissions, escalation, human review, or another independent source of evidence.

The goal is not to remove the gate. The goal is to avoid confusing failure of the gate’s recognition mechanism with failure of the thing being examined. That is exactly the distinction the Claude test exposed.

We have built the same problem into work

The parallel to hiring becomes stronger, not weaker, once we admit that gates can have good reasons to exist. Hiring systems face real risks. Applicants sometimes misrepresent experience. Credentials sometimes matter. Regulated work may require specific qualifications. Recruiters and managers have finite time. Organizations cannot investigate every claim without structure, and some form of screening is unavoidable.

The criticism therefore cannot simply be “stop gatekeeping.” The better question is:

What does the system do when a legitimate person cannot traverse its preferred verification path?

A capable worker may lack the expected title. A career may not fit standard chronology. Relevant experience may use different vocabulary. Transferable capability may come from another domain. A resume may not contain the keywords an automated screen expects. An internal employee may possess useful capability that is invisible to the formal job architecture.

If the system says only recognized or rejected, uncertainty gets destroyed. “Our primary mechanism could not establish this person’s capability” silently becomes “this person does not possess the capability.” Those claims are not equivalent.

A more disciplined system would preserve an intermediate state:

Unable to establish through the primary recognition pathway.

That state can justify more evidence, another verification route, bounded review, or a human decision. It does not require blindly admitting every claim. It simply refuses to pretend that non-recognition is proof of nonexistence.

The false negative is often invisible

This remains one of the hardest properties of the problem. When a dangerous source gets through a security control, something may happen that exposes the failure. When a bad hire occurs, organizations may notice. When fraud succeeds, there is often a loss to investigate.

When a legitimate source never enters an AI model’s context, nothing happens. When a capable applicant never reaches a hiring manager, nothing happens. When an internal expert is never considered for a problem, nothing happens.

The system records the objects it admitted. It often has very little information about the valuable objects it excluded. That makes legitimate false negatives structurally difficult to measure. A system can become increasingly optimized around the population that successfully traverses its gates while learning very little about the population the gates make invisible.

This is why the cost of exclusion has to be designed into the architecture rather than waiting to appear naturally in the data.

A better trust path does not mean a weaker one

The obvious response to a false negative is sometimes to loosen the control. That is not necessarily the right response. If the threat is real, weakening the primary boundary may simply create a different and more serious failure.

The more interesting design pattern is an alternate trust path:

primary path fails
→ preserve uncertainty
→ do not automatically admit
→ do not automatically invalidate
→ seek alternate verification
→ constrain access where appropriate
→ add evidence
→ escalate when justified
→ make a bounded decision

That pattern applies surprisingly well across security, AI systems, hiring, professional capability, and organizational knowledge. It also explains why evidence and provenance matter so much. Alternate verification is only useful if the system can distinguish what was observed, what was supplied, what was inferred, what remains unknown, and who has authority to make the next decision.

The failure belonged to more than one layer

After further investigation, the Claude incident can be decomposed much more precisely.

OSI-PIA had a discoverability problem. The repository was public but did not reliably surface through the open-web search paths tested. Claude’s tested environment had a source-admission constraint. A user-supplied URL that had not appeared through its available search path could not simply be fetched. Anthropic has documented a serious adversarial environment. Its September 2026 threat report describes sophisticated malicious users attempting to conceal intent, fragment harmful activity, circumvent safeguards, and use Claude across cyber, surveillance, weapons, fraud, and other harmful workflows. Claude preserved the epistemic boundary reasonably well. It did not claim to have inspected a source it had not actually read and ultimately declined to invent a project interpretation.

Those are four different findings. None should be silently collapsed into another. The SEO problem is mine to improve. The tool boundary belongs to the tested Claude environment. The threat environment helps explain why conservative controls may be necessary. The model’s refusal to pretend it had evidence is a separate behavioral result.

That decomposition is more useful than blaming either side.

The broader principle

The three articles in this mini-series began with a tiny failure and ended somewhere much larger. The False Negative Stack examined how layered recognition systems can make legitimate capability disappear. Access Is Not Authority separated reachability, admissibility, understanding, epistemic authority, permission, and consequence. This final piece adds the uncomfortable part: sometimes the gate producing the false negative is defending against a threat that is entirely real.

That does not invalidate the gate. It creates a design obligation.

Trustworthy systems must manage both the danger of admitting what should be excluded and the cost of excluding what should be admitted.

That principle applies to machine security, hiring, organizational knowledge, access control, credentialing, fraud prevention, and many other systems that decide what or who becomes visible enough to receive trust.

The question is not whether we can build a world without gates. We cannot. The question is whether our gates are sophisticated enough to know the difference between dangerous, disproven, unknown, and simply not yet recognized.

A public repository gave us a small example. Human work systems are already living with the consequences at much larger scale.