Offline pre-computation · conversational surface

Inventory Intelligence Platform

The clearest instance of the pattern that runs through most of my work: build the offline pipeline that makes the question cheap, then put a conversational agent on top of it. Here the economics are unusually explicit: scanning a multi-million-row weekly history at query time burns both latency and tokens, so the scan moves offline and the agent reads one pre-computed row. The second decision is where the compute runs: not in the warehouse, because the warehouse is a shared, already-degraded, business-critical resource, and adding calculation views to it would make my feature someone else's outage.

2025–2026platformdataeconomicsagentic

What I owned

Sole author of the pipeline, the statistics, and the conversational agent on top of it.

History window pre-computed: 104 weekly versionscodeHistory window pre-computed
Source columns read: ~50codeSource columns read
Agent lookup per history question: 1 rowcodeAgent lookup per history question

3decisions, argued

Built onAirflow on Cloud ComposerFastAPI on Cloud Runpandas and numpySAP HANALangGraph with a deep-agent framework

01Problem 02My role 03Decisions 04Impact

The problem

What was actually wrong

The current inventory snapshot was already answerable. History was not. Questions like how excess changed quarter over quarter, whether coverage is improving, or how much of a change is price versus volume all required scanning a weekly-history table with millions of rows.

Putting that scan behind a conversational agent makes it worse, not better: the agent would need to author and run the scan on every question, paying both the query latency and the token cost of reasoning over the result.

Before this existed

Ad-hoc analyst queries against the weekly history table, one question at a time.

Constraints

The conditions the design had to hold under

Constraints are the interesting part of an architecture. Without them any diagram looks reasonable.

The warehouse is shared and already slow: It is business-critical and used by many teams. Any design that adds calculation views or stored procedures to it exports my compute cost onto everyone else.
Query-time scanning costs tokens, not just seconds: An agent that scans at query time pays latency and then pays again to reason over a large result inside a context window.
The current snapshot path must not be touched: The latest weekly snapshot was already served directly and worked. Scope was historical statistics only.

My role

Sole author of the pipeline, the statistics, and the conversational agent on top of it.

Designed

The pre-computation architecture and its scope boundary
The statistics surface: trends, price-versus-volume attribution, excess against target, coverage and weeks of supply, lead-time decomposition, forecast accuracy, down-level risk and aging

Built

A weekly pipeline that reads a slimmed history, computes statistics off-warehouse, and rebuilds the target table
A self-contained conversational agent with its own chat interface and no dependency on any other repository

Led

Established the pipeline-then-agent pattern that later repeated across mode shift, transit-time prediction and excess-and-obsolete

Decisions

The calls I would defend

Each one with the alternatives I rejected, what the choice cost, and how it turned out.

Decision

The warehouse is read from and written to. All computation happens in a service outside it.

Context

The obvious answer is to push computation to the data as calculation views or stored procedures.

Alternatives, and why not

Calculation views in the warehouseIt is a shared instance that is already slow and business-critical. Adding my compute to it means my feature's cost lands on every other team, and my feature's regression becomes their outage.

Rationale

Move the data to the compute when the compute is yours and the data store is everyone's. The transfer cost is bounded and predictable; the shared-resource contention is neither.

What it cost

Data transfer on every run, mitigated by reading only the most recent 104 weekly versions and only the columns the statistics actually need.

Outcome

The pipeline can be tuned, retried and scaled without a warehouse change request.

Impact

What changed, and how it is known

Every figure carries its basis. Nothing here is rounded up, and nothing modelled is presented as a result.

104 weekly versionscode

History window pre-computed

~50code

Source columns readDeliberately slimmed from the full table to keep transfer and memory small.

1 rowcode

Agent lookup per history question

Business: History questions that required an analyst and a warehouse query are answered conversationally, in seconds.
Engineering: The pattern proved reusable: the same pipeline-then-agent shape was applied to mode-shift optimisation, transit-time prediction and excess-and-obsolete analytics, and this agent was later absorbed into the larger platform as a domain.

Leadership and hindsight

What I influenced, and what I would change

Technical leadership

Built it standalone and self-contained so it could be adopted without adopting a platform, then let it be absorbed once the platform existed, designing for absorption rather than for territory.

What I would do differently

Weekly freshness is right for history and was never re-examined as the questions got more operational. Some of what users now ask would justify a daily incremental path.
The statistics surface grew by request. It would benefit from being described declaratively, the same semantic-layer argument that applies to the wider platform.

Go deeper

The detail, for people who want it

Collapsed by default. The case study stands without any of it.

Why this became the portfolio's spine

The same shape appears in four systems: an offline pipeline that makes a question cheap to answer, and a conversational agent on top of it as a domain in a shared platform.

Inventory statistics became an inventory domain agent. A mode-shift engine became a mode-shift agent. A transit-time model became a data source inside the platform. Excess-and-obsolete analysis became an excess-and-obsolete agent.

The difference between building some agents and defining how an organisation turns an analytics asset into a conversational product is exactly this repetition, and the fact that each one was designed to be absorbed rather than to stand alone forever.

Stack

What it is built on, and why that

A technology list without reasons is a list of things I have heard of.

Airflow on Cloud Composer: Weekly scheduling with authenticated invocation.The pipeline is a single scheduled call to a service, so scheduling and compute stay independently operable.
FastAPI on Cloud Run: The build-statistics endpoint doing the actual computation.Compute lives here rather than in the warehouse, which is the central decision of the design.
pandas and numpy: The statistics themselves.The slimmed input fits comfortably in memory, so the simple tool is the correct one.
SAP HANA: Source history and the rebuilt statistics table.Read from and written to, never computed in.
LangGraph with a deep-agent framework: The conversational surface, with its own bundled chat interface.Self-contained by design, with no dependency on any other repository, so it could be run and evaluated on its own.

4 more sections are written and hidden: constraints, leadership, deeper, stack.