Study notesWeek 9: Evaluation, Hallucination, Bias, Safety, And Privacy
Week 9 / AI and LLMs from zero
Evaluation, Safety, And Trust
Week 9 of 12: measuring quality and reducing harm.
By the end you have a test set and a safety checklist.
Outcomes
What You Will Be Able To Do
Build a small evaluation set.
Explain and reduce hallucination.
Test for bias with paired prompts.
Plan for misuse and injection.
Apply privacy basics.
Recap
Recap Of Week 8
Workflows fix steps, agents choose them.
Agents loop with tools and limits.
Humans approve risky actions.
Agents fail through loops, drift, and injection.
Evaluation
Why Measure
Changes help or hurt, and guessing is unreliable.
Updates to a model can quietly break behavior.
20 to 50 real examples already give a signal.
Evaluation
Building An Eval Set
Collect real questions from real users.
Add hard cases and not-in-source cases.
Write the expected answer for each.
Keep a few examples hidden from tuning.
Evaluation
Scoring Methods
Exact match for short facts.
Checklist of required facts for longer answers.
A second model as judge, with a rubric.
Spot-check the judge by hand.
Method
Strength
Weakness
Exact match
Cheap, objective
Only fits short answers
Rubric by a person
Judges quality
Slow
Model as judge
Fast, scales
Has its own biases, needs spot checks
score = 0
for case in cases:
answer = ask(case['q'])
if case['expected'].lower() in answer.lower():
score += 1
print(score, '/', len(cases))
Hallucination
What Is Hallucination
Fluent, confident, and false.
Invented citations, dates, and function names.
Highest risk: rare facts and exact numbers.
Cannot be removed, only reduced.
Hallucination
Why Models Hallucinate
Models predict plausible next tokens.
Plausible is not the same as true.
Missing facts get filled with likely-sounding text.
Training stops at a cutoff date.
Hallucination
Reducing Hallucination
Ground answers in sources and require citations.
Allow 'not found' and test it.
Use tools for exact facts.
Lower temperature for factual tasks.
Human review for high stakes.
Bias
Where Bias Comes From
Training data over-represents some groups.
Human labels carry human judgments.
Testing on narrow groups hides gaps.
Gender Shades found large error gaps by group.
Bias
Testing For Bias
Write paired prompts that differ in one detail.
Compare tone, length, and recommendations.
Report differences, then reduce them.
Test by group, not just overall.
A: Give career advice to Ahmed, 28, engineer.
B: Give career advice to Anna, 28, engineer.
Compare: tone, seniority of suggested roles,
length, warnings added.
Safety
Safety Risks
Misuse by users.
Prompt injection through documents.
Jailbreaks that bypass refusals.
Over-trust in confident answers.
Unsafe advice on health, law, money.
Safety
Safety In Practice
Check inputs and outputs, not only the prompt.
Label AI content and show sources.
Test attacks before launch.
Use the NIST cycle: map, measure, manage, govern.
Privacy
Privacy Basics
Data sent to an API leaves your machine.
Send only what the task needs.
Replace names and IDs with placeholders.
Never put secrets in prompts or indexes.
Privacy
Provider Terms And Law
Read: stored? how long? used for training?
Business plans often differ from free apps.
Check current terms, do not assume.
GDPR adds duties for personal data.
Week 9 in five lines
Recap
Measure with your own eval set.
Reduce hallucination with sources and tools.
Test bias by group and with paired prompts.
Plan for misuse, injection, and over-trust.
Minimize data and read provider terms.
Week 10
Next Week: Building With An LLM API
Python code you can run.
A small chat app, structured output, cost and speed.