Build the engine, not the feature

Sales reps kept hitting the same wall mid-conversation: a customer asks something specific about a product, and the answer is buried somewhere in a manual nobody can read on the spot. An R&D analyst, since the organisation had a capable ML team, had built a chatbot prototype that answered those questions from the product documents, and the sales team already liked it in pilot. But it lived in a notebook with a quick UI bolted on. Useful, and nowhere near production-ready, secure, or reusable. The ask was to take it the rest of the way.

We didn’t invent the chatbot and we didn’t write the production code. The prototype was the analyst’s, and the build was a vendor team’s. Our seat was in the middle and above: take the prototype, define what “production” actually required, make the one design decision that shaped everything, own the security, and direct the build.

From the outset we asked the team to build for generality. A chatbot is a general interaction model; it would fit far more than one use. So rather than a point tool for sales, we specified a modular system with a reusable core. That was the call, made before there was any second use to justify it. It paid off twice over: the same codebase later answered ESG sections of tender RFPs and generated product slide decks.

The easy path was a tool for sales. We built the engine instead, and got three products for the price of one.

Generality only works if the seam is in the right place. Almost everything lived in a reusable core, and each use was a thin layer of routing and prompts on top. The core carried the chat UI, streaming responses, chat-history management, document chunking, search, logging, request processing, file handling, and shared authorisation, over a clean set of integrations. Add a new use, and you write the thin part, not the engine.

Every answer ships with its source, and it’s worth being precise about why. This wasn’t a guardrail against hallucination so much as a way to let the rep verify. The instinct came from the analyst’s ML background, wanting answers quantified and checkable, with a confidence score and a test set behind them. In use, it became an ergonomics win: open the cited source and it scrolls straight to the page the answer came from, so the rep can read the surrounding context and trust it before repeating it to a customer.

The hard part of retrieval-augmented generation is almost never the model. It’s getting clean text out of messy documents. Product material is full of diagrams with embedded text and tables with footnotes, and naive extraction mangled them. The fix was a parsing pipeline: pull the images out first, run them through Document Intelligence to recover the text, then vectorise. For the footnoted tables, we used a specialised OCR-backed parser where ordinary parsing failed.

The generality stopped being theoretical when the second and third uses arrived. The ESG use case came in independently but was structurally the same: query the documents, answer with sources. So it simply reused the existing implementation. The deck generator was an enhancement the sales team asked for: build a presentation for a given product topic. That one needed a new technique on top of the same core, hierarchical prompting, which generates an outline first, then expands each point into slides.

Azure wasn’t a contrarian bet here. It was where the work already lived: the model was Azure OpenAI, the prototype was already deployed there, and the documents were in SharePoint, with no dependency on anything elsewhere. The path of least resistance was also the right one.

The security detail worth naming is the unglamorous kind that’s invisible when it’s right. We validated every authentication token on the backend, against the identity provider’s (Microsoft Entra’s) published public certificate. We didn’t trust the client, and didn’t skip the signature check. It’s the sort of thing a less experienced build does loosely and never notices, until someone forges a token.

This one earned its numbers. It’s in production and adopted by the sales team. All product documentation is indexed and kept current, every answer carries a citation that opens to the exact page, and answers return instantly. One honest caveat we’ll keep in: indexing isn’t instant. A new document flows from SharePoint through Document Intelligence on a scheduled trigger, minutes to hours behind if the queue is busy. Because the core was built to be reused, the same system went on to draft ESG tender sections and generate product decks: three applications from one codebase.

Two patterns outlive this build. The first: a working prototype and a production system are different animals. The distance between them, security, integration, reliability, reuse, is exactly the work that turns a promising demo into something an organisation depends on. The second, and the one to carry forward: when you’re handed a prototype that solves one problem, the highest-leverage question isn’t “how do we ship this?” It’s “what is this generally?” Answer that well once, and the second and third uses cost a thin layer instead of a new build.

A few durable ideas held the whole thing together.

• →Build the engine, not the feature. A reusable core outlives any single use and pays for itself on the second.
• →Productionizing is its own discipline. A notebook that demos is not a system that runs; someone has to own the gap.
• →Make verification effortless. A citation that jumps to the exact page lets the human check in seconds.
• →The RAG wall is parsing. Clean text out of messy documents is most of the battle; the model is downstream of it.
• →Take the constrained path when it's right. Azure here wasn't a bet, it was where the work already lived.
• →Own the invisible security. Validate tokens on the backend against the issuer's certificate, before anyone forges one.