LOCAL EVIDENCE WORKSPACE · APPLIED AI ENGINEERING
What does a successful AI run actually prove?
AI Workflow Reliability Console makes source outcomes, supporting evidence, and comparison limits visible in one local workspace.
- Stage
- Local application · reviewed local candidate
- My role
- Project owner; agent-assisted development
- Built with
- Python · SQLite · plain JavaScript
Problem
A workflow can report success while leaving its reviewer unsure what completed, which checks passed, or whether the evidence supports the claim. A single status hides those different questions.
Engineering question
How can a reviewer distinguish an imported claim from supporting evidence, then compare two runs without changing what is being evaluated?
What I built
A local application that imports supported evidence bundles, inspects outcomes and checks, compares compatible records, explains refused comparisons, and exports exact JSON or Markdown reports. The run list and inspection area keep the selected evidence close to the decision a reviewer needs to make.
Actual local captures
Architecture
Standard-library Python handles validation and a loopback-only server. SQLite retains imported bytes and their revisions; plain JavaScript presents the inspection and comparison views. The interface and exports share the same evidence core. No model runtime or external service is needed to inspect the supplied examples.
Evaluation
The accepted local walkthrough covered import, inspection, compatible comparison, refusal, and export. A separately reviewed local successor corrects nanosecond timestamp ordering while preserving the accepted interface.
The recorded candidate check passed 87 Python test methods, including 14 HTTP checks, with 44 conformance vectors exercised. The verification record documents the Linux temporary-path adaptation and independent review scope. These checks do not establish a new browser acceptance.
In owner review, I asked for a clearer explanation of source-reported Success beside Independently verified: No. The revised explanation says the report claims the task worked, while a separate trusted check has not confirmed it. That distinction was then clear enough to accept the walkthrough. This was qualitative owner feedback.
Guardrails and boundaries
Imports are data: their commands, paths, and URLs are not executed or fetched. Source claims stay distinct from the console's assessment. Comparison requires compatible workflow, task, scope, configuration, and check sets. Byte consistency helps preserve a record; it does not establish its author or the truth of its claims.
Failure evidence
An incompatible comparison returns its reason without an improvement score. An imported success claim with incomplete support remains unverified. These are visible outcomes a reviewer can inspect and export, rather than missing details behind an overall success label.
Tradeoffs
A compact local application makes this first review workflow easier to inspect and reproduce. Keeping collection and model execution outside the console also makes its responsibility clear: assess the supplied evidence and explain the comparison boundary. Live collection and broader integrations require further work.
Current limitations
The console serves one local operator; browser acceptance was on macOS. It has no hosted service, account authentication, or runtime collector. General evidence-format interoperability remains partial, and imported review claims do not become independent verification. The captures use synthetic or preserved evidence.
What comes next
Extend the supported evidence sources while keeping source claims, verification, and comparison compatibility distinct. The useful product question remains how much explanation a person needs to make the next judgment confidently.
Related work
ARMS explores the other side of this evidence workflow: a bounded code repair with protected checks and a separate review decision.