Skip to content
Kiran
← Work

A decision tree an analyst can read, in YAML

Carrier Root-Cause Validation Agent

The domain logic is declarative rather than prompted. Eighteen structured knowledge files hold the business glossary, carrier ontology, column semantics, cross-reference rules, service-level rules, issue categories, output taxonomy and a genuine typed decision tree with an entry-point filter and reason-code branching. The agent follows the tree; the tree is reviewable by an analyst who cannot read Python. A deterministic router and pre-check run before the model, so the model only sees cases that need judgement.

2025–2026agenticevaluationdata

What I owned

Owner. Agent, declarative knowledge model, deterministic pre-routing and the evaluation harness.

Structured knowledge files
18codeStructured knowledge files
Batch entry points
2codeBatch entry points
Built onLangGraph with Claude on VertexStructured YAML knowledge modelSAP HANA
01Problem02My role03Decisions04Impact

The problem

What was actually wrong

Carrier-reported root causes for delivery misses drive downstream reporting and carrier performance conversations. When the codes are wrong, both are wrong.

Validating them requires cross-referencing milestone data, service-level rules and free-text comments, judgement work at a volume that made manual validation partial at best.

Constraints

The conditions the design had to hold under

Constraints are the interesting part of an architecture. Without them any diagram looks reasonable.

The rules belong to operations, not engineering
Categories, service-level rules and the routing logic change on an operational timeline. If they live in prompts, every change is an engineering ticket.
Ground truth exists and must not leak
An analyst-labelled workbook is available for evaluation. The production path must be structurally incapable of seeing that column.

My role

Owner. Agent, declarative knowledge model, deterministic pre-routing and the evaluation harness.

Designed

  • The declarative knowledge model: eighteen structured files including a typed decision tree
  • Two separate entry points so the production path cannot see the ground-truth column

Built

  • The agent, the deterministic router and pre-check, the knowledge generator, and both batch entry points

Decisions

The calls I would defend

Each one with the alternatives I rejected, what the choice cost, and how it turned out.

Decision

Eighteen structured knowledge files define the glossary, ontology, column semantics, cross-reference rules, service-level rules, categories, output taxonomy and the decision tree the agent follows.

Context

The rules are owned by operations and change on their timeline.

Alternatives, and why not

  • Encode the rules in the system promptEvery rule change becomes an engineering change, and nobody outside engineering can audit what the agent is actually applying.

Rationale

A typed decision tree with an explicit entry point, routing and declared output columns is reviewable by the person who owns the logic. The agent's job is to follow it, not to reconstruct it from prose.

What it cost

A knowledge model to maintain and generate, which is more machinery than a prompt.

Outcome

The same instinct (move domain knowledge out of prompts into a declarative, reviewable artefact) appears independently in three of my systems, which is how I know it is a principle rather than a preference.

Impact

What changed, and how it is known

Every figure carries its basis. Nothing here is rounded up, and nothing modelled is presented as a result.

18code
Structured knowledge files
2code
Batch entry pointsProduction, which cannot see the label column, and evaluation, which can.
Business
Carrier root-cause codes are validated at volume rather than sampled, so downstream reporting and carrier conversations rest on checked data.
Engineering
A declarative knowledge model that operations can review, with deterministic pre-routing and evaluation against human expert labels.

Leadership and hindsight

What I influenced, and what I would change

Technical leadership

  • Put the rules where the people who own them can read them, which turned rule changes from engineering tickets into edits.

What I would do differently

  • Throughput was never instrumented. I have an impression of how fast a batch runs and no measurement, which is exactly the kind of claim I would refuse to accept from someone else.
  • The knowledge files are generated as well as hand-edited, and the boundary between generated and authored content is not as clear in the repository as it should be.

Stack

What it is built on, and why that

A technology list without reasons is a list of things I have heard of.

LangGraph with Claude on Vertex
The validation agent following the declarative decision tree.The judgement step is genuinely a judgement; everything around it is deterministic.
Structured YAML knowledge model
Glossary, ontology, semantics, rules, taxonomy and decision tree.Reviewable by the operations analysts who own the logic.
SAP HANA
The production feed of carrier failures and milestone data.Read-only, and structurally separated from the labelled evaluation workbook.

3 more sections are written and hidden: constraints, leadership, stack.

Next