Guide to AI production readiness
Make the production decision about the complete system.
Bring the business sponsor, domain reviewers and operators together around a specific configuration. Review the task, its dependencies and the people who will respond when the system needs attention.
What should a team review before putting an AI workflow into operation?
Review representative task results, information and access boundaries, connected actions, failure handling, capacity and cost. Confirm the deployed configuration, operating owners, monitoring and recovery plan. Record the evidence, unresolved items and acceptance decision. A model result alone cannot establish whether its complete business workflow is ready.
Create one release review record.
For each area, name the owner, link the evidence, record any unresolved items and enter an acceptance decision. The rows below identify what to collect. They do not report completed tests.
Task and business quality
Owner: task sponsor. Evidence: reviewed representative cases. Unresolved item: unmet acceptance criteria. Decision: accept, narrow the scope or hold.
Information and actions
Owner: data and system owners. Evidence: source, permission and action tests. Unresolved item: failed boundary or completion check. Decision: approve the tested scope or hold.
Reliability and operating fit
Owner: technical lead. Evidence: dependency-failure, workload and cost results. Unresolved item: untested limit or failure. Decision: accept the operating range or hold.
Operation and recovery
Owner: service operator. Evidence: alerts, escalation and recovery exercises. Unresolved item: missing owner or unproven recovery step. Decision: approve the plan or hold.
Release and later changes
Owner: release approver. Evidence: configuration, test versions and change process. Unresolved item: missing approval or regression result. Decision: release the named version or hold.
Test the work people will actually rely on.
Use representative cases and define the expected result before testing. Include normal work, missing information, exceptions and actions the system must refuse. Check the resulting business record when the task changes another system.
Illustrative task: prepare a service-response draft for human review. Test the following situations against the brief.
- A complete request: the draft uses the relevant policy and business record.
- Missing information: the draft identifies the gap instead of inventing a fact.
- Restricted information: the system does not expose a record outside the permitted scope.
- An unavailable dependency: the failure reaches the assigned review path.
- A repeated request: the configured process avoids an unintended duplicate action.
Review quality separately from activity.
Monitoring can show that a request ran, how long it took and which resources it used. Evaluation asks whether its result met the business criteria. Review both: a completed execution can still produce a poor draft or an incorrect action.
Compare results with the baseline, retain the tested configuration and use domain reviewers where judgement is needed. Set quality, workload and cost thresholds for this task instead of borrowing a universal target.
Exercise the response to a problem.
Name who responds to alerts, investigates incomplete work, approves changes and restores service. Walk through a dependency failure and the intended recovery, including the responsibilities of other suppliers or your own team.
Keep testing and review active after deployment. NIST's AI Risk Management Framework links evaluation and monitoring with accountable roles, incident response and change management.
Record a decision with a clear boundary.
Approve a named configuration for a defined scope, or record what prevents approval. Assign each unresolved item an owner and next action. Keep the evidence with the decision so later model, knowledge, tool or workflow changes can trigger the relevant review.
Questions and answers
It is one part of the evidence. Test knowledge, tools, permissions, workflow completion, deployment and operating responsibilities as a complete system.
No. It is a planning and acceptance record for a specific workflow. It does not certify legal compliance or replace your required reviews.
Your operating model should name the accountable approver and required business, technical and other reviewers before the decision.
Review relevant cases after changes to models, prompts, knowledge, tools, workflows or operating conditions, and use production findings to update the test set.