Design for AI dependency failure before launch
Define timeouts, retries, degraded behavior, queues, human fallback, observability, and user communication for an unreliable dependency.
Direct answer
An AI response can be late, unavailable, malformed, unsafe, or plausible but wrong. Reliability planning must cover both infrastructure failure and unacceptable model behavior. The product needs a bounded fallback that does not turn one dependency problem into a wider outage.
Decision sequence
- Set a request deadline and a maximum retry policy that avoids retry storms and duplicate actions.
- Define which work can queue, which can degrade, and which must stop for human review.
- Monitor request success, latency, useful completion, unsafe or irrelevant output, and fallback use.
- Test provider, region, quota, retrieval, and tool failures before the workload becomes critical.
Evidence to keep
- A service objective tied to the user task.
- A tested degraded mode or manual route.
- Alerts with named response ownership.
- A record of dependency and model-behavior incidents.
Sources and interpretation boundaries
Google Cloud Architecture CenterAI and ML perspective: ReliabilityReliability goals, modular design, observability, model behavior, and end-to-end operations.Amazon Web ServicesGenerative AI LensLifecycle review across scoping, model selection, integration, deployment, and continuous improvement.National Institute of Standards and TechnologyAI Risk Management FrameworkVoluntary risk-management structure and the Govern, Map, Measure, and Manage functions.
Decision boundary
This guide is an original educational synthesis. It does not inspect your workload, validate a contract, test a provider, certify security, recommend an investment, or promise cost, performance, funding, revenue, savings, or growth.