AI Development

AI development is the work of turning a bounded decision, search, generation, classification, or workflow problem into a controllable product capability. The model is one component. A dependable system also needs lawful and usable data, a defined task, measurable acceptance criteria, permissions, deterministic rules, retrieval or tool boundaries, human review, fallback behavior, monitoring, and an owner for incidents and change.

This service is for teams that can identify a valuable job where probabilistic software may improve speed, coverage, consistency, or access to information. It is not a promise that every workflow needs a model. Some problems are better solved with clearer forms, search, business rules, analytics, or ordinary automation. Discovery should earn the AI architecture by comparing those options against the same user outcome and risk.

Start with the task, not the model

A useful brief names who is making a decision, what information they have, what output is required, what happens next, and the cost of an incorrect, missing, delayed, or fabricated result. “Add AI” is not a testable requirement. “Help a support agent find the approved policy passage and draft a response that must be reviewed before sending” creates an observable task, an authority boundary, and a way to evaluate usefulness.

Task suitability depends on tolerance for variation. Low-consequence assistance, summarisation, discovery, drafting, and prioritisation can often accommodate review and uncertainty. Consequential decisions involving access, employment, credit, health, safety, legal status, pricing, payments, or irreversible actions require stronger controls and specialist review. An AI component should not quietly become the final authority simply because its output reads fluently.

Resolve data rights before designing the pipeline

The data inventory should identify sources, owners, permitted purposes, sensitivity, residency, retention, quality, update frequency, deletion obligations, and whether content may be sent to an external provider. Public availability does not automatically establish permission for training, indexing, or reuse. Client documents, user conversations, operational records, licensed datasets, and generated outputs may each have different contractual and privacy boundaries.

The implementation should separate development examples, evaluation sets, retrieval sources, production inputs, logs, and any material considered for model improvement. Access should follow the least privilege needed for the task. Secrets, personal information, confidential documents, and unreviewed production content should not leak into prompts, traces, analytics, or support exports. Where a provider processes data, its current terms, deployment region, retention controls, and training settings need explicit review.

Build a baseline before selecting an AI approach

A baseline reveals whether the proposed system is actually better than the current process or a simpler alternative. It may be a manual sample, keyword search, a rules engine, a conventional classifier, or the existing workflow. The baseline should use representative cases and record quality, time, failure categories, escalation burden, and cost in the terms that matter to the buyer. Without it, a polished demonstration can be mistaken for evidence of production value.

Evaluation examples should reflect ordinary cases, difficult cases, incomplete inputs, conflicting sources, unsafe requests, out-of-scope requests, and known edge conditions. Labels need a definition and, where judgement is involved, a review method. Test data should be separated from prompts and tuning work so the team does not optimise to a remembered answer set. The objective is a repeatable comparison, not a single favourable output.

Choose rules, retrieval, models, and agents deliberately

Use deterministic rules where the answer must be exact

Eligibility gates, permissions, required fields, monetary calculations, workflow transitions, and other explicit business constraints usually belong in ordinary code or a reviewable rules layer. A model may explain a rule or help classify an input, but it should not invent the governing rule. Keeping deterministic authority outside the model improves testing, auditability, and safe fallback.

Use retrieval when the answer depends on approved knowledge

Retrieval-augmented generation can ground an answer in selected documents, but retrieval alone does not guarantee correctness. The system needs an ingestion boundary, document identity, permissions, chunking and indexing choices, update and deletion behavior, citation or provenance display, and a response when evidence is absent or contradictory. Retrieval quality and answer quality should be evaluated separately so the wrong document is not hidden by confident prose.

Use a model when probabilistic interpretation adds value

A model can help interpret language, images, audio, or mixed inputs; classify ambiguous material; transform content; or draft outputs. Selection should consider task quality, modality, context limits, structured-output behavior, supported regions, privacy controls, latency, reliability, and cost. Provider benchmarks are useful context, but acceptance must be measured on the product’s own representative cases and current model version.

Use an agent only when controlled tool use is necessary

An agent adds planning and tool execution across multiple steps. That flexibility also expands the failure surface: it may choose the wrong tool, repeat an action, act on stale context, exceed cost limits, or attempt something outside the user’s intent. Tool permissions, argument validation, idempotency, spending limits, step limits, confirmation gates, and complete action traces should be designed before an agent can affect external state.

Design human review as a real operating role

“Human in the loop” is incomplete unless the reviewer has the context, authority, time, and interface to make a meaningful decision. The workflow should show the source material, generated output, uncertainty or reason for escalation, prohibited actions, and an understandable record of what will happen after approval. Review queues need ownership, priority, ageing, reassignment, and a safe state when no reviewer is available.

Feedback buttons are not automatically an evaluation system. Feedback should connect to a defined case, model and prompt version, retrieved evidence, user role, and outcome. Sensitive material needs appropriate access and retention. Corrections may improve prompts, rules, retrieval, training data, or the surrounding product; the team should diagnose the failure before treating every correction as model-training data.

Permissions and safety belong around every capability

The application should enforce who may submit data, retrieve a source, view an output, approve a recommendation, invoke a tool, or inspect a trace. The model should receive only the context and capabilities allowed for that user and task. Hiding an action in the interface is not authorization. Server-side checks remain responsible for identity, scope, resource ownership, rate limits, and consequential changes.

Safety controls should follow plausible misuse and failure, not a generic filter checklist. Relevant controls may include input validation, source allowlists, output schemas, policy checks, prompt-injection resistance, content moderation, confidential-data handling, tool restrictions, and escalation. The system should define which responses are refused, which are safely limited, which require review, and how users recover without being encouraged to bypass controls.

Fallback behavior is part of the product

Models, retrieval services, vector stores, and external tools can be unavailable, slow, rate-limited, or changed by their providers. A product needs a response for each dependency failure. Depending on the task, it may show verified source results without generation, route work to a manual queue, preserve a draft, retry safely, use a tested secondary provider, or stop the action. A fallback should not silently lower quality or remove a required safety boundary.

Uncertainty also needs a product response. If the system lacks evidence, receives conflicting material, or produces an invalid structured output, it should not manufacture completion. The interface can ask for missing information, cite the available sources, explain that the task could not be completed, or escalate to a person. Honest incompleteness is often safer and more useful than a confident guess.

Budget latency and cost across the whole request

User-perceived latency includes authentication, retrieval, reranking, prompt assembly, model generation, tool calls, validation, storage, and interface updates. The team should establish a budget for the complete journey and test representative input sizes and concurrency. Streaming can improve perceived responsiveness, but it does not correct a slow or unsafe workflow and may complicate moderation, cancellation, and partial-output handling.

Cost should be measured per completed business task, not only per token. It can include embeddings, search, model calls, retries, tool usage, storage, observability, human review, and provider minimums. Limits should exist by user, workspace, workflow, and environment where appropriate. The product should expose abnormal use and runaway loops before they become a financial or operational incident.

Make changes measurable and reversible

Prompts, model identifiers, retrieval settings, tools, policies, evaluation sets, and output schemas should be versioned. A change can improve one class of cases while degrading another, so release evidence should compare the candidate against the accepted baseline and important slices. Staged exposure, feature flags, shadow evaluation, and rollback paths can reduce the impact of an unexpected regression.

Monitoring should cover product outcomes and system behavior. Useful signals may include completion and escalation rates, invalid outputs, unsupported claims, retrieval misses, safety interventions, latency distributions, cost per task, tool failures, review ageing, and user-reported problems. Exact thresholds follow the use case. Dashboards are only useful when an owner knows what action to take when a signal changes.

Prepare for AI incidents and provider change

An incident plan should cover harmful or incorrect output, confidential-data exposure, unauthorized tool use, provider outage, compromised credentials, unexpected cost, and a material quality regression. The team needs a way to disable a feature or tool, preserve relevant evidence, notify responsible owners, correct affected state, and document the decision to restore service. Logs must support investigation without becoming a second uncontrolled copy of sensitive data.

External models and APIs evolve. Model versions can be retired, behavior can change, pricing and limits can move, and regional availability can differ. The architecture should isolate provider-specific code, record the model used for important outputs, and retest material changes. Multi-provider support is valuable only when the alternative is tested against the same task, data, safety, latency, and cost requirements.

Accept the system with evidence, not a compelling demo

• The task, users, authority boundary, prohibited actions, and escalation path are documented and reflected in the working product.
• Representative, difficult, unsafe, and out-of-scope cases are evaluated against a named baseline with recorded failure categories.
• Data sources, rights, permissions, retention, provider processing, deletion, and production logging boundaries are reviewable.
• Rules, retrieval, model calls, tools, human review, validation, fallback, latency, and cost behave as specified in the target environment.
• Monitoring, feature controls, incident ownership, rollback, known limitations, and deferred work are included in release evidence.

What the client receives at handover

The signed agreement defines the applicable application source, orchestration and prompt configuration, retrieval pipeline, schemas, evaluation assets, tests, deployment material, monitoring, documentation, and administrative controls. Handover should distinguish client data and deliverables from provider services, open-source components, licensed material, reusable frameworks, and credentials. It should also identify the current model versions, data dependencies, known limitations, cost assumptions, and owner for future changes.

The result should be a product capability the client can inspect, operate, measure, constrain, and evolve. Engineering can improve readiness and produce evidence; it cannot responsibly guarantee perfect answers, uninterrupted third-party service, legal or regulatory suitability, provider behavior, cost at every scale, user adoption, or a commercial outcome. Those boundaries make the proposal more credible and the operating system safer.