Courses / AI Integration For Companies / Week 11

Week 11 of 12 · Intermediate

Scaling And Risk

You learn how to keep AI systems safe and reliable as use grows: monitoring, human oversight, incident response, vendor lock in and model change management. You write a monitoring plan and an incident playbook for one use case.

3 hours18 slides6 min read

Open slides Read study notes

What Changes When You Scale Monitoring: Know Before The Customer Does Logging And Drift Human Oversight That Works Incident Response Vendor Lock In Model Change Management

You Will Be Able To

Choose monitoring measures for quality, safety, cost and use
Design human oversight that matches the risk of a use case
Write an incident playbook with roles, steps and a pause switch
Assess vendor lock in and plan an exit path
Manage a model change with testing, staged release and rollback

What Changes When You Scale

A pilot has a few users, a watchful owner and a narrow scope. Scaling adds more users, more data, more edge cases and less attention per case. Problems that were rare at fifty users happen daily at five thousand. Costs, errors and legal exposure grow with use, so the controls from weeks 5 and 10 must now run as routines and not as one off checks.

Scale one step at a time. Move from one team to a few, then to a division, and pass a gate at each step: results meet the success measures from week 6, monitoring is in place, the playbook is tested, and the owner agrees. If a gate fails, stay where you are and fix it.

Monitoring: Know Before The Customer Does

Monitoring means measuring a running system continuously and alerting a person when something moves outside a set range. Language models can degrade quietly, because their answers stay fluent even when they become wrong. So monitor more than uptime.

Watch four groups. Quality: accuracy on a fixed test set that you run regularly, sampled human review of real outputs, and user ratings. Safety: blocked or flagged outputs, personal data appearing where it should not, and attempts to misuse the system. Cost: tokens and spend per use case and unit cost, as in week 9. Use: active users, requests, response time and error rate. Set thresholds in advance and decide who receives each alert.

Quality: regular test set, sampled review, user ratings
Safety: flagged outputs, data leaks, misuse attempts
Cost: spend per use case and unit cost
Use: users, volume, response time, error rate

Monitoring Plan Example

Group	Measure	How	Alert to
Quality	Test set accuracy	Weekly automated run	Product owner
Quality	Sampled review pass rate	50 cases per week	Product owner
Safety	Flagged outputs	Filter log	Risk contact
Cost	Cost per resolved ticket	Billing tags	Business owner
Use	Error rate and response time	Service metrics	Technical owner

Every measure has a threshold and a named recipient.

Logging And Drift

Keep logs of inputs, outputs, model version, and reviewer actions, with access controls and retention that respect KVKK and GDPR. Logs let you investigate an incident, answer an audit, and improve the system. Decide what you must mask or avoid storing, since a log full of personal data is a new risk.

Drift is a change in the world or in the data that makes a once good system worse. New products, new policies, new customer language and seasonal patterns can all move the input away from what you tested. Re run your test set on a schedule, add new real cases to it, and compare results over time. A drop is a trigger to investigate, not a reason to ignore the system.

Test Set Accuracy Over Time With Drift

percent correct (illustrative)

Illustrative: a regular test run shows the slide months before users complain.

Human Oversight That Works

Human oversight means a person can understand, question and override what the system does. In week 7 you placed the human: approve everything, sample, or review flagged cases. At scale this needs care, because reviewers tire. If a person approves hundreds of drafts a day, approval becomes a click, which is called automation bias: the tendency to trust a machine suggestion without checking it.

Keep oversight real. Show the reviewer the source next to the output. Include known wrong examples in the review queue as a test of attention. Limit the daily volume per reviewer, give them authority to reject without penalty, and track how often reviewers change outputs. A reviewer who never changes anything may be a signal of trust or of tiredness, so check which. For high risk uses under the EU AI Act, human oversight is a legal requirement and not an option, so confirm the requirements with your counsel.

Incident Response

An incident is any event where an AI system causes or may cause harm: a wrong answer sent to many customers, personal data exposed, a biased result, a security breach, or a cost spike. Prepare before it happens. Write a playbook of one or two pages and rehearse it once a year.

A good playbook has these parts: how to detect and report, a severity scale, who leads the response, how to pause or switch off the system, how to contain harm, whom to notify, how to communicate with affected people, and how to learn afterwards. Notification duties can be strict. A personal data breach under GDPR generally has to be reported to the authority within 72 hours of becoming aware, and KVKK has its own breach notification rules, so involve legal and your data protection contact from the first hour. After the incident, hold a blameless review, record the cause and fix the system, not only the symptom.

Detect and report: one clear channel, open to all staff
Severity scale: who is called at each level
Pause switch: tested, with a named person allowed to use it
Notify, communicate, review, and record the fix

Incident Response Flow

Detect and report
Pause if needed
Assess severity
Contain and notify
Review and fix

The pause switch comes before the full investigation.

Vendor Lock In

Lock in means that switching away from a vendor is so costly that you cannot realistically do it. It grows from proprietary formats, prompts tuned to one model, data stored in a vendor's system, special features that nothing else offers, and contracts with long terms and high exit fees. Some lock in is acceptable when the benefit is clear. The risk is to be locked in without noticing.

Reduce it with design choices. Keep prompts, test sets and business rules in your own repository. Put an abstraction layer between your application and the model so a different model can be called with small changes. Store your data in open formats you control. Run your test set against a second model every quarter, so you know what a switch would cost. Negotiate export rights and reasonable notice for price and service changes. Know what you would do if the vendor raised prices sharply, changed terms or shut down a model.

Model Change Management

Vendors update models, retire old ones and sometimes change behaviour without a change of name. A new version can be better on average and worse on your task, because your prompts and checks were tuned to the old one. Treat a model change like any other production change.

Follow a routine. Pin to an exact model version where the vendor allows it. When a new version appears, run your full test set and compare. Review the differences by hand on the cases that matter most. Release to a small share of users first, watch the monitoring measures, then widen. Keep the previous version available for rollback until the new one proves stable. Record the change, the test results and the approver. Track announced deprecation dates, so retirement of a model is a planned project and not a surprise.

Cost belongs in the scaling conversation too. Usage often grows faster than planned once a tool proves useful, and a monitoring dashboard that shows spend next to quality helps the owner decide when growth is healthy. Agree a monthly review in which the product owner, the technical owner and finance look at the same page. If unit cost rises while quality falls, pause expansion and investigate before adding more users.

Key Terms

Monitoring: Continuous measurement of a running system with alerts when results leave an agreed range.
Drift: A change in data or the world that makes a once good system perform worse.
Human oversight: The ability of a person to understand, question and override what an AI system does.
Automation bias: The tendency to trust a machine suggestion without checking it.
Incident playbook: A short written plan that says who does what when an AI system causes harm.
Kill switch: A tested way to pause or turn off a system quickly.
Vendor lock in: A situation where switching away from a vendor is too costly to be realistic.
Staged release: Rolling a change out to a small share of users first and widening step by step.
Rollback: Returning to the previous working version after a change causes problems.

Exercise: Monitoring Plan And Incident Playbook

Choose one use case and list two measures each for quality, safety, cost and use.
Set a threshold for each measure and name who receives the alert.
Write a severity scale with three levels and the person called at each.
Describe the pause switch: who may use it and how fast it works.
List five parts of your system that tie you to one vendor and one step to reduce each.
Write the steps you would follow to test and release a new model version.