An SLA for an AI application needs the classic elements of availability, support hours, response and resolution times, plus AI-specific clauses: measurable quality on an agreed evaluation set, handling of model updates and deprecations, dependencies on third-party model providers, data residency and cost controls. Without these, the contract covers whether the system is up but not whether it still works correctly.
On this page
A traditional service level agreement answers a simple question: is the system available, and how fast does the provider react when it is not? For AI applications, that is necessary but not sufficient. An AI feature can be fully available and still deteriorate, for example after a model update, a change in user behaviour or new types of input. It also depends on third-party model providers whose terms the service provider does not control. This article explains which clauses an SLA for an AI application should contain and how to make them measurable.
The classic foundation
Every SLA starts with the familiar components, and AI applications are no exception. Define availability as a percentage over a period, with a clear definition of what counts as unavailable and which maintenance windows are excluded. Define support hours and channels. Classify incidents by severity, for example critical when the application is unusable for all users, and agree response and resolution times per class. Add reporting duties and the consequences of missed targets, such as service credits or escalation rights. These elements are well established; the challenge is to extend them to the parts that are specific to AI.
Quality as a service level
The key addition is a measurable quality level for AI features. The practical approach is an agreed evaluation set: a versioned collection of representative inputs with expected outcomes, owned jointly by client and provider. The SLA then defines a threshold, for example a minimum share of correctly classified cases, and how often the set is run, such as after every change and at least monthly. If quality drops below the threshold, it is treated as an incident with its own severity and resolution time. In production, add a live quality signal where possible, such as the share of AI suggestions users accept or the rate of escalations to humans, and agree on alert thresholds. This turns "the AI got worse" from a vague complaint into a measurable event.
Model changes and deprecations
Model providers regularly release new versions and retire old ones, sometimes with limited notice. The SLA should state who monitors provider announcements, how quickly a deprecated model must be replaced, and that any model change passes the evaluation set before going live. It should also define whether the client must approve model changes, which may matter for regulated use cases or where outputs are documented. Pinning model versions where providers allow it, and testing replacements in a staging environment, should be contractual obligations rather than good intentions.
Third-party dependencies
Most AI applications depend on external model APIs, vector databases or hosting providers. The service provider cannot promise more availability than these dependencies offer unless the architecture compensates, for example with a fallback provider or a degraded mode that keeps the application usable without AI. Be explicit: list the third-party services, state whether their outages count against the SLA, and define the expected behaviour during such outages. A central AI platform with multiple model providers, such as KalaidoCreate, can make fallback strategies easier to implement and to contract.
Data, security and costs
AI applications process data in more places than classic software, so the SLA or the accompanying data processing agreement should fix where data is processed and stored, which model providers receive which data, how long prompts and outputs are logged and who can access the logs. Include security incident notification times consistent with your GDPR obligations. Usage-based model costs can also surprise both sides. Agree on monitoring of costs per period, alert thresholds and what happens when usage exceeds the forecast, such as notification before additional costs arise or automatic throttling.
Checklist of SLA clauses
- Availability target, measurement method and maintenance windows.
- Support hours, severity classes, response and resolution times.
- Evaluation set, quality threshold, test frequency and quality incidents.
- Live quality monitoring with alert thresholds.
- Model update, deprecation and approval process.
- List of third-party dependencies, fallback behaviour and how their outages count.
- Data residency, logging, retention and security incident notification.
- Cost monitoring, thresholds and escalation.
- Reporting, review meetings and consequences of missed targets.
Making it work in practice
An SLA is only as good as the monitoring behind it. Before signing, ask the provider to show how they measure availability, run evaluation sets and track model changes; our guide on monitoring AI-built apps describes the tooling. Plan regular reviews, quarterly for most applications, where both sides look at incidents, quality trends and upcoming provider changes. The SLA also belongs in a broader maintenance plan, discussed in maintaining an AI-generated app after launch. Process maturity on the provider side matters here, which is why ISO-certified development partners are often preferred for business-critical AI applications.
OnlineAppCreator offers SLAs for the products it builds and operates, including for B2B and public sector clients; the details are described on our SLA and public sector page. Because we see our role as taking products from idea to operation, the SLA is designed together with the architecture rather than added at the end, and it is part of our custom development and company building work.
Frequently asked questions
Can an AI application's accuracy be guaranteed in an SLA?
Not absolutely, but a minimum quality level on an agreed evaluation set can be contracted, with defined actions and resolution times when results fall below it.
Who is responsible when the model provider has an outage?
That depends on the contract. The SLA should list third-party dependencies, state whether their outages count and define fallback behaviour during them.
How often should AI quality be measured under an SLA?
After every change to prompts, models or data and at regular intervals, typically at least monthly, plus continuous live signals where available.