Skip to content

Guide · Artificial intelligence

AI Development Guide

Most AI features fail on data access, weak evaluation or missing human review rather than on the model itself. This guide walks a product lead through the choices that decide whether an AI feature earns trust once real users rely on it.

Evaluation planned before any prompt is written

Human review designed into the workflow

Inference cost treated as a product metric

Before you build

Decisions to make before you build an AI product.

01Decide

The job the model is allowed to do

Write down whether the system drafts, classifies, retrieves, recommends or acts. A model that suggests text for a person to approve carries far less risk than one that sends messages or changes records on its own.

02Decide

Which data it may read

List every source the feature touches, who owns it, whether consent covers this new use and whether personal or confidential records would leave your infrastructure when calling a hosted model.

03Decide

Hosted model, open weights or both

Hosted APIs are quick to start and improve without effort on your side; open-weight models give control over data residency and cost at volume but add serving, patching and monitoring work.

04Decide

What counts as a good answer

Agree a written definition of correct, acceptable and harmful output for your domain, with real examples, before anyone tunes a prompt. Without it, quality arguments become matters of taste.

05Decide

Who is accountable for mistakes

Name the person or team who reviews flagged output, handles complaints and can switch the feature off. Oversight that belongs to nobody quietly disappears after launch.

Structure

How an AI product is structured.

01Layer

Data and retrieval layer

Connectors, cleaning, chunking and an index (often a vector store alongside ordinary search) that supplies the model with current, permission-filtered context instead of relying on what it memorised in training.

02Layer

Model and prompt layer

Versioned prompts, model selection, tool definitions and structured output schemas, kept in source control so every change can be compared against the evaluation set.

03Layer

Guardrail layer

Input checks for prompt injection and out-of-scope requests, output checks for policy, personal data and format, and fallbacks that return a safe response when a check fails.

04Layer

Evaluation and observability layer

A test set scored automatically on each change, tracing of production calls with inputs, retrieved context and outputs, and dashboards for latency, refusal rate and cost per task.

05Layer

Human oversight layer

Review queues, confidence thresholds that route uncertain cases to people, feedback buttons that feed the test set, and an audit trail that shows who approved what.

Planning sequence

A planning sequence for an AI product.

  1. 01Step 1

    Collect real examples first

    Gather a few hundred genuine inputs from the workflow you want to improve, with the answer an expert would give. This becomes the first evaluation set and exposes the hard cases early.

  2. 02Step 2

    Prove it on paper with a baseline

    Try the simplest approach (a single prompt with retrieved context) against that set, and record where it fails. Many teams discover the problem is missing data rather than a weak model.

  3. 03Step 3

    Design the review loop

    Decide what a person sees, what they can correct and how corrections are stored. Build this before widening access, because it is how quality improves later.

  4. 04Step 4

    Release to a narrow group

    Ship to internal users or one customer segment behind a feature flag, compare outcomes with the old process and watch traces daily for surprising behaviour.

  5. 05Step 5

    Widen access with budgets in place

    Set per-user rate limits, spending alerts and a model fallback order, then expand only when evaluation scores and review findings stay stable.

Mistakes

Mistakes that derail an AI product.

01Avoid

Judging quality by demo

A handful of impressive answers in a meeting says nothing about the long tail. Without a scored test set, a prompt change can fix one case and silently break ten others.

02Avoid

Retrieval that ignores permissions

An assistant that searches every document can show one customer or employee material they were never allowed to see. Filtering must happen before context reaches the model.

03Avoid

Letting the model act unchecked

Giving an agent write access to email, payments or records without approval steps turns a wrong answer into a real-world incident that is hard to undo.

04Avoid

Ignoring cost per task

Long contexts, retries and chained calls multiply spend. Teams that never measure cost per completed task find the feature uneconomic only after it is popular.

Start from a workflow, not a model

The strongest AI products begin with a specific, repeated task that people already do badly or slowly: triaging support tickets, summarising long case files, matching invoices to orders. Naming the task tells you what inputs exist, what a good output looks like and who will notice when it goes wrong.

Starting from a model tends to produce a chat box looking for a purpose. Users try it, get a mixed experience and stop. A narrow feature that removes one painful step usually earns more trust than a general assistant, and it gives you a clear measure of success.

Data decides the ceiling

A model can only be as useful as the context it receives. Before design work, audit the sources: are they current, are they structured enough to search, and who is allowed to see which parts? Messy permissions are the most common reason enterprise assistants stall before launch.

Consent and purpose matter too. Records collected to run a service may not be usable for a new automated feature without telling people or updating your lawful basis. Settle this with whoever owns data protection early, because rebuilding a pipeline later is far harder than designing for it.

Evaluation is the real product work

An evaluation set is a collection of real inputs with expected outcomes, scored automatically or by reviewers. It turns vague debate about whether the assistant feels better into a number you can track release by release, and it catches regressions when a provider updates a model.

Score more than accuracy. Track groundedness, refusal when information is missing, tone, format compliance, latency and cost. Keep adding production failures to the set, so the hardest cases you have seen are tested on every future change.

Designing human oversight that lasts

Oversight works when it is part of the interface rather than a policy document. Show reviewers the source passages behind each answer, make correction faster than rewriting from scratch and route low-confidence cases to them automatically.

Measure the reviewers’ experience as well. If they approve nearly everything without reading, the check has become theatre; if they rewrite most outputs, the feature is not ready. Both signals should feed decisions about where automation is allowed to expand.

Security risks specific to language models

Language models follow instructions found anywhere in their input, which means a malicious web page, email or uploaded file can try to redirect them. Treat retrieved content as untrusted, keep tool permissions minimal and require confirmation for any action with side effects.

Leakage runs the other way as well. System prompts, other users’ data and secrets in context can surface in answers. The OWASP list of risks for large language model applications is a practical checklist to review with your engineers before launch.

What drives cost and effort

Effort concentrates in data preparation, evaluation tooling and review interfaces rather than in the model call itself. Integrations with older internal systems and permission models often take longer than the visible feature.

Running cost depends on context length, call volume, model tier and how many steps each task needs. Caching repeated context, routing simple requests to smaller models and capping retries are the usual levers, and each should be measured against your evaluation scores so savings do not quietly reduce quality.

FAQ

Questions to resolve before the build.

01Do we need to train our own model?

Rarely at the start. Most product teams get further by retrieving the right context and writing clear instructions for an existing model. Fine-tuning becomes worth considering when you have a stable task, a large set of labelled examples and evidence that prompting has plateaued.

02How do we stop the model making things up?

You reduce it rather than eliminate it. Ground answers in retrieved sources, ask for citations, refuse when context is missing and measure unsupported claims in your evaluation set. For high-stakes output, a person should check before anything reaches a customer.

03What regulation applies to an AI feature?

It depends on where you operate and what the feature decides. Data protection law such as the GDPR applies whenever personal data is processed, and the EU AI Act sets duties that scale with risk. Take legal advice for your jurisdiction; this guide is not legal advice.

04How should we price an AI feature?

Start from measured cost per completed task, not per call, then decide whether to absorb it, meter it or bundle it into a higher plan. Usage caps protect margins while you learn how heavily customers rely on it.

05When is an AI agent appropriate?

When the task genuinely needs several steps across tools and the cost of a mistake is low or reversible. Start with the agent proposing actions for approval, and remove the approval step only for actions your evaluation shows it handles reliably.

Primary sources

References behind this page

Dated official documentation, standards, and research that support the factual claims on this page.

  1. 01
  2. 02
  3. 03

How to read this page

Scope, cost and timeline figures are planning guidance until confirmed in a written proposal. Diagrams and screens are illustrations, not client deliverables, and source-code rights and handover follow the signed agreement. See the delivery process and the full disclaimer.

Next decision

Turn the brief into an accepted product scope.

Define outcomes, constraints, evidence, rights and handover before delivery begins.

Commercial rights, repositories, environments, documentation, acceptance and handover remain contract-defined.