# How I put AI into a delivery system Source: https://manaienko.com/how-i-work/ai-in-the-delivery-system/ Author: Artem Manaienko I build shared workspaces with repository context, a knowledge base and a workflow for each delivery stage. Engineers review the outputs at the points where a wrong decision would be costly. ## My principles 08 operating rules 1. [01 Start with the cost of understanding the system](#practice-ai-in-the-delivery-system-principle-1) 2. [02 Build a shared workspace](#practice-ai-in-the-delivery-system-principle-2) 3. [03 Keep human review at each gate](#practice-ai-in-the-delivery-system-principle-3) 4. [04 Put access boundaries in the tooling](#practice-ai-in-the-delivery-system-principle-4) 5. [05 Prove it with the engineers](#practice-ai-in-the-delivery-system-principle-5) 6. [06 Evaluate on a bounded sample](#practice-ai-in-the-delivery-system-principle-6) 7. [07 Count the whole attempt](#practice-ai-in-the-delivery-system-principle-7) 8. [08 Choose the model for the task](#practice-ai-in-the-delivery-system-principle-8) ### Start with the cost of understanding the system Before component implementation, several engineers had to reconstruct existing behaviour and agree a contract. I used AI there first because the repeated investigation cost real time. ### Build a shared workspace Repository context, architecture, constraints and task-specific workflows belong together. I want the team to reuse that environment across delivery stages. ### Keep human review at each gate Engineers review proposed contracts before implementation. Tests and normal pull-request review remain mandatory on every production change; model review adds another check. ### Put access boundaries in the tooling The workspace carries access and confidentiality restrictions. Platform engineers had no access to protected health data, and the tooling needs to preserve that boundary. ### Prove it with the engineers I used the workspace myself, then with two pilot engineers. After roughly ninety minutes of paired onboarding per person, I rolled it out to the team. Their earlier poor experience with AI made that practical proof necessary. ### Evaluate on a bounded sample Deterministic filters remove obvious noise before paid model calls. I compare prompt results on the same sample and put usage and cost beside them. ### Count the whole attempt Price per token alone misses how many calls and tokens a model needs to finish. I record the route, usage and attributable cost for each attempt, together with elapsed time. ### Choose the model for the task Planning and independent review stay on the stronger profile. Bounded implementation can use the cheaper tier with deterministic gates. Extra waiting costs less in an unattended run than when an engineer is pairing with the agent. ## What I built and tried 04 cases ### We kept doing the same component work Teladoc Health · mid-2025 onward **Situation:** Integrating design-system components into our Server-Driven UI meant alignment calls, repetitive implementation, code review, then production triage. We had roughly 52 newer components and 50+ legacy ones. Documentation was often missing, so engineers had to compare the specification with the Android, iOS, React and Ruby implementations before agreeing what to build. Discovery alone took two hours or more with several people. Integration took another two to four engineering days. I wanted to get this routine out of their way. **What I did:** - I built a shared workspace with repository, GitHub and Figma context, plus a Markdown knowledge base of architecture, constraints and integration rules. - I gave each stage its own workflow. Before discovery, the model compared the specification with existing implementations and proposed a shared contract. Engineers reviewed it before implementation. - The workflows also covered implementation, validation, PR checks and triage. Normal tests and human review stayed in place. Access and confidentiality restrictions were built into the workspace; platform engineers had no access to protected health data. - I used it myself first, then with two pilot engineers. Onboarding took roughly 1.5 hours of pairing per person. I later helped other teams adopt it through mentoring. **Outcome:** The discovery meeting came down to at most thirty minutes. Analysis and agreement took around four hours in total. Roughly 12 engineers used the workspace daily and about 20 in a typical month. I estimated approximately 3x productivity and about 120 developer-hours saved per month across the integration workflow. These are historical estimates. I did not instrument the baseline or preserve the calculation model for the 3x and 120-hour figures. The four hours cover analysis and agreement, not implementation across every platform. How a component moved through the workspace 1. 01 ### Read the specification Compare it with released behaviour and find gaps before the meeting. 2. 02 ### Agree the contract Propose shared properties and interfaces, keeping platform-specific constraints. Human review 3. 03 ### Update the backend Represent the agreed contract in the Ruby API core. 4. 04 ### Implement per platform Android, iOS and React, with Storybook and demo-app integration. 5. 05 ### Validate and triage Check against the specification and identify the likely source of defects. Human review 6. 06 ### Review the release Check release classification, PR content and commit scope. Mandatory PR review Engineers reviewed the proposed contract before the implementation workflows used it. ### Generating screens did not solve the integration Teladoc Health · 2025 **Situation:** I wanted to take this further and generate screens using our existing components. I passed Figma context into Cursor, and also tried Builder.io and Figma Make. **What I did:** - I compared generated screens with the components we actually shipped. - I checked runtime-component usage, navigation, backend state and validation against existing flows. **Outcome:** The tools could reproduce styles and tokens. But they did not reliably use our runtime components. Navigation, state, validation and legacy integration still needed manual work, which was most of the job. We stopped the experiment after reorganisation and headcount reductions. I cannot say how much further we would have got. ### I wanted to compare prompts properly Independent product · 2026-present **Situation:** In my current private, pre-launch AI product, a prompt change affects both output quality and cost. One nicer answer was not enough to tell me whether to keep the change. **What I did:** - I built an evaluation flow that runs candidate prompts and filter settings on a bounded sample from real sources. Deterministic filters remove obvious noise before paid model calls. - I put token use and model cost beside the results. Evaluation stays separate from product data, and settings change only when I explicitly save them. **Outcome:** I can compare prompts on the same sample and see the cost before keeping a change. I do not yet have enough runs to claim an improvement in quality or cost. ### The cheaper model took twice as long Independent product · 2026 **Situation:** I wanted to know whether a cheaper model would actually save money on implementation. It cost roughly a tenth per token, but could spend much longer finding its way through a task. I ran the same specification at the same reasoning effort on both tiers. **What I did:** - I compared tool calls, tokens, elapsed time and estimated cost for the two runs. - Afterwards, I kept planning and independent review on the stronger profile and used the cheaper tier for bounded implementation. Each attempt recorded its route, usage and cost. **Outcome:** The cheaper tier made 2.64x as many tool calls, used 3.35x the tokens and took about twice as long. Its estimated token bill was still roughly a third. That made it useful for an unattended implementation run, where I could afford to wait. One task, one specification. Calls, tokens and time were measured; cost came from list prices, not an invoice. I did not count review rounds or rework. Both runs had to pass the gates, but that alone does not establish equal quality or a general rule for model selection. The two implementation runs Stronger tier ### About 30 minutes - 123 tool calls. - 13.97M tokens. - Roughly ten times the price per token. Cheaper tier ### About 62 minutes - 325 tool calls. - 46.74M tokens. - Estimated token bill about a third. **~3x** cheaper token bill, estimated from list prices This comparison covers the implementation runs. Review and rework costs were not measured. What I wish I had measured earlier At Teladoc, I reconstructed the savings after changing the workflow. Next time I would keep a baseline and a repeatable set of specifications, bugs and review comments from the start, including false positives and rework.