Models
decide.
Code acts

How I engineer AI systems that a business can actually run on: a lightweight shared integration core, a durable agent harness with human approval as a first-class state, and independent applications on top. Built to be observable, recoverable, testable and safe to operate.

  1. 01Design thesis
  2. 02Architecture
  3. 03Anatomy of a run
  4. 04Core components
  5. 05Security model
  6. 06Proof: Agent Workbench
  7. 07Module matrix
  8. 08Decisions
  9. 09Roadmap

01 Design thesis

Narrow the
model’s job

Most failures in production AI come from giving a model too much freedom and too little structure. Every system here follows five rules.

  1. 01

    Constrain outputs to typed choices.

    The model selects from validated, schema-defined options: which tool, which fields, which widget. Deterministic code compiles and executes those choices. In the analytics module the model never writes SQL, never writes UI code and never types the numbers on screen.

  2. 02

    Treat everything the model touches as untrusted.

    Model output, retrieved documents, emails and tool responses are data, never instructions. Authority comes from the policy layer and from humans, not from text in the context window.

  3. 03

    Make runs durable, not conversational.

    A run is a persisted state machine with checkpoints, not a chat transcript held in memory. It can pause for days waiting on approval, survive a deploy and resume without repeating side effects.

  4. 04

    If it isn’t measured, it doesn’t work.

    Every run emits a trace with token usage, latency and cost. Behaviour is pinned with scripted-model tests and regression datasets, so model or prompt changes are evaluated, not eyeballed.

  5. 05

    Share the integration layer, not the application.

    A small common core handles providers, tools and observability. Each application keeps its own frontend, data model and deployment. No monolith, no framework built ahead of real use cases.

02 Architecture

Six layers.
One boundary

Requests flow down through the stack. Nothing below the permission boundary is reachable without passing the policy layer, whatever the model asks for.

L6Surfaces
  • Ops consoleapprovals, traces, cost, failures
  • Business appsportals, internal tools
  • Assistant UIschat, generative dashboards
  • Inbound channelsemail, webhooks, forms, schedules
L5Applications
  • Workflow automation
  • Knowledge assistant
  • Document processor
  • Analytics agentbuilt
  • Research system
  • Marketing studio
L4Agent harness
  • Run loopturns, tool dispatch, stop conditions
  • Context managerassembly, budgets, compaction
  • Run storestate machine, checkpoints, event log
  • Approval queuepause · edit · approve · reject
  • Schedulerjob queue, retries, cancellation
L3Integration core
  • Provider adaptersone interface, native features kept
  • Tool registryschemas, side-effect class, metadata
  • Tracing & costspans, tokens, latency, spend
  • Eval hooksoutput checks, sampling to datasets
L2Policy
  • Permission boundaryper-tool scopes · least-privilege credentials · approval rules · budgets · rate limits · audit log
L1Resources
  • Model providersOpenAI · Anthropic
  • Business systemsemail, CRM, calendar, accounting
  • Datasemantic layer, retrieval index, document store

What’s shared

L2 to L4. Provider adapters, the tool registry, tracing, policy and the durable runtime are the parts every application needs and nobody should rebuild.

What isn’t

Frontends, data models, databases and deployments. Each application at L5 is independently useful and independently deployable, usually on its own subdomain.

Extended, not modified

New capabilities like retrieval, document parsing or a business API arrive as registered tools with schemas and a side-effect class. The core doesn’t change to accommodate them.

03 Anatomy of a run

One enquiry,
eight steps

Follow a single inbound quote request through the harness. Step through it, or let it play. Illustrative payloads, real design.

A webhook creates a durable run.

The inbound email is normalised into an event and persisted before any model is called. The message ID becomes the idempotency key, so a duplicate webhook delivery maps to the same run instead of a second one.

POST /runs
{
  "workflow": "enquiry.triage",
  "trigger":  { "type": "email.received", "id": "msg_8c1e…" },
  "idempotencyKey": "msg_8c1e…"
}
→ 201 { "run": "7f3a", "status": "queued" }

04 Core components

What’s in
the core

The reusable pieces, and the decisions inside them. Advanced packages are extracted only when a real project needs them.

Provider layer

One interface for model calls across providers, without flattening them into a lowest common denominator.

  • Adapters for the OpenAI Responses and Anthropic Messages APIs
  • Structured outputs and tool calling from shared schemas
  • Streaming, usage reporting and normalised errors
  • Provider-native features stay reachable when needed
  • Model configurable per role, e.g. composer vs reviewer

Tool registry

Tools are typed, discoverable units with explicit risk, not functions the model happens to be able to call.

  • Zod input and output schemas, exported as JSON Schema
  • Side-effect class: read, write, external, irreversible
  • Scoped credentials resolved server-side at call time
  • Validation errors returned to the model as tool results
  • MCP-compatible interfaces where useful

Durable execution

Work that outlives a single request, a deploy or a human’s lunch break.

  • Runs as persisted state machines with an append-only event log
  • Checkpoint per step; resume without re-running side effects
  • Idempotency keys on every external write
  • Retry policies with backoff; cancellation and timeouts
  • Approval pauses that hold state, not processes

Retrieval

Grounding answers in a business’s own documents, and only the ones the asker may see.

  • Ingestion pipeline with document versioning and re-indexing
  • Embeddings plus keyword search for hybrid retrieval
  • Permission filters applied at query time, not after
  • Citations down to the passage; “not found” is a valid answer
  • Retrieval accuracy measured against labelled questions

Observability

Every decision the system makes can be inspected after the fact.

  • Span-level traces for model turns, tool calls and approvals
  • Token usage, latency and approximate cost per run
  • Failure classification: model, tool, policy, upstream
  • Sensitive payloads redacted by default, logged by opt-in
  • Replayable recordings of complete runs

Evaluation

Measuring whether the system does its job, before and after every change.

  • Scripted fake-model tests for loop and tool sequencing
  • Golden tests per output type and schema
  • Regression datasets sampled from real runs
  • Tool-call accuracy and task success rates
  • Model and prompt comparisons on cost and quality

05 Security model

Threats in.
Controls out

Never allow a model-generated command, query or request to run merely because the model asked. Here’s how each class of risk is handled.

ThreatControlWhere it’s enforced
Prompt injection via an email, web page or documentInbound and retrieved content is labelled as data; authority comes only from policy and humans; consequential tools require approval regardless of what the context says.context manager · policy
Model-written queries reaching the databaseThe model fills a validated query spec; a semantic layer compiles it to parameterised SQL over allowlisted fields, with row limits and timeouts.semantic layer
Over-broad access to business systemsPer-tool, least-privilege credentials held server-side and resolved at call time. The model never sees a key.tool registry · policy
Hallucinated figures in reportsWidgets are hydrated from stored query results, never from model text. Numbers quoted in AI-written insights are checked against results and flagged if unmatched.hydration
Duplicate side effects on retryIdempotency keys per run and step; resume from checkpoints rather than replaying the run.durable runtime
Runaway cost or abuse of public demosPer-run token ceilings, daily budgets that survive restarts, per-visitor rate limits, concurrency caps and a file-based kill switch.usage guard
Sensitive data leaking into logsRedaction by default; payload logging is opt-in per workflow; secrets never in repos, CI logs or build artefacts.tracing · repo hygiene

All code is written on the assumption it may be published: .env.example files with placeholders, server-side auth checks, ownership validation on every resource, and no privileged infrastructure reachable from public demos.

06 Proof: the first module, built

Agent
Workbench

The analytics module, built end to end as a standalone application. Ask a market-research question in plain English; an agent queries real data and composes a dashboard while you watch. It’s where the core’s patterns were proven first.

request path as built
  1. questionplain English, follow-ups edit the view
  2. agent loop2 tools: query_dataset, compose_view
  3. semantic layerallowlisted measures, dimensions, joins
  4. DuckDBparameterised SQL over Parquet
  5. hydrationwidgets filled from stored rows
  6. SSE → Reactwidgets stream in, full trace on demand
~7.8MIowa retail sales rows, 2022–2025, aggregated month × store × product
~126KSteam games with genres, tags, prices, reviews and playtime
18widget types, from KPI cards to a county choropleth
$0AI cost to replay recorded runs with their original timing

Constrained generative UI

The model picks registered widgets and maps columns from query results by ID. A broken widget is dropped; the rest of the view survives.

Number verification

Figures quoted in AI-written insights are matched against the underlying results. Anything that can’t be matched is flagged on screen.

Guarded live mode

Per-visitor rate limits, a persisted daily token budget, per-run ceilings, concurrency caps and a kill switch that needs no restart.

Tested without AI

Query compiler, hydration and the agent loop are covered by tests with a scripted fake model. No API calls, no cost, deterministic.

07 Module matrix

Every module,
its hard part

Each application exists to demonstrate a distinct engineering problem, not just a feature list.

ModuleKey techniquesThe hard problemStatus
Agent harness & ops consoleorchestration, durable state, approval workflows, tracing, cost monitoringAgents that are recoverable, inspectable and safe to leave runningFlagship · in development
Analytics agentsemantic layer, tool calling, generative UI, streamingNumerically correct answers without letting the model touch SQL or figuresBuilt
Workflow automationwebhooks, classification, structured extraction, approvalsRetries and error recovery across third-party APIs without duplicatesPlanned
Knowledge assistantingestion, embeddings, hybrid retrieval, citationsPermission-aware retrieval over documents that changePlanned
Document processorPDF and image ingestion, multimodal extraction, review UIInconsistent layouts, extraction confidence and reliable approvalPlanned
Research systemplanning, parallel tool calls, source management, synthesisClaim verification over long-running work; delegation only where it helpsPlanned
Marketing studiobrand context, multi-stage generation, content validationConsistent, controllable output with editors in the loopPlanned
Full-stack business appauth, role-based permissions, relational modelling, API designA reusable, well-tested application architecture; no AI requiredPlanned

08 Decisions & trade-offs

What I chose,
and why

Short decision records. Each one has a cost, and that cost is accepted on purpose.

ADR-01

A shared core, not a platform.

Chose: a small integration layer shared by independent apps.

Over: one application hosting every demo, or a framework built up front.

Cost: some duplication across frontends. Worth it to keep each app shippable alone.

ADR-02

Typed specs over generated code.

Chose: the model fills validated specs that deterministic code compiles.

Over: letting the model write SQL, UI components or scripts.

Cost: less open-ended flexibility. Every new capability needs a schema.

ADR-03

One loop before many agents.

Chose: a single tool-calling loop as the default.

Over: multi-agent orchestration by default.

Cost: fewer impressive diagrams. Delegation is added only where it measurably beats the simpler design.

ADR-04

Provider-agnostic, not provider-blind.

Chose: a common adapter interface with an escape hatch to native features.

Over: an abstraction that hides what makes each provider good.

Cost: adapters carry provider-specific code paths and tests.

ADR-05

Approval is a state, not a popup.

Chose: human approval modelled in the run state machine with checkpoints.

Over: synchronous confirmation inside a request.

Cost: durable storage and resume logic from day one.

ADR-06

Replay-first public demos.

Chose: recorded runs replayed at zero cost, with a guarded live mode.

Over: an open live endpoint paid for by every visitor’s curiosity.

Cost: visitors see curated questions first; live questions are rationed.

09 Roadmap

Shipped in
phases

The full reference system will run on its own subdomain. It’s built in phases, each shipping something that works on its own.

  1. Phase 1 · Proof

    First module live

    Agent Workbench, the analytics module, deployed as a standalone app with replayed and guarded live runs.

    Live
  2. Phase 2 · Commercial core

    Practical AI capabilities

    Workflow automation, knowledge assistant and a full-stack business app. The shared core is extracted here.

    Next
  3. Phase 3 · Harness

    Agent harness & ops console

    Secure tool execution, persistent state, tracing, approvals, evaluation, recovery.

    Flagship
  4. Phase 4 · Specialise

    Advanced applications

    Research and reporting, document processing, marketing studio, extended analytics.

    Planned

10 Talk shop

Let’s build
something
reliable

Hiring, collaborating, or evaluating whether I can build this for your business? I’m happy to walk through the architecture, the code and the trade-offs in detail.